Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added publishing logic. Includes javadoc fixes necessary for building. #5

Merged
merged 2 commits into from
Jan 21, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,23 @@
./gradlew build
```

## Code style

[Spotless](https://github.com/diffplug/spotless/tree/main/plugin-gradle) is used for code formatting.

To auto-format your code, you can run:
```shell
./gradlew spotlessApply
```

## Local development

If you are doing local development and want to consume `deephaven-csv` changes in other components, you can publish to maven local:

```shell
./gradlew publishToMavenLocal -x signMavenJavaPublication
```

## Benchmarks

To run the all of the [JMH](https://github.com/openjdk/jmh) benchmarks locally, you can run:
Expand Down
2 changes: 2 additions & 0 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -63,3 +63,5 @@ spotless {

check.dependsOn(jmhClasses)
assemble.dependsOn(jmhJar)

apply plugin: 'io.deephaven.csv.java-publishing-conventions'
17 changes: 17 additions & 0 deletions buildSrc/build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
plugins {
id 'groovy'
id 'groovy-gradle-plugin'
}

java {
toolchain {
languageVersion = JavaLanguageVersion.of(11)
}
}

repositories {
mavenCentral()
maven {
url "https://plugins.gradle.org/m2/"
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
plugins {
id 'java'
id 'signing'
id 'maven-publish'
}

java {
withJavadocJar()
withSourcesJar()
}

def developerId = 'deephaven'
def developerName = 'Deephaven Developers'
def developerEmail = 'developers@deephaven.io'

def projectUrl = 'https://github.com/deephaven/deephaven-csv'
def orgName = 'Deephaven Data Labs'
def orgUrl = 'https://deephaven.io/'

def licenseName = 'The Apache License, Version 2.0'
def licenseUrl = 'https://www.apache.org/licenses/LICENSE-2.0.txt'

def issuesSystem = 'GitHub Issues'
def issuesUrl = 'https://github.com/deephaven/deephaven-csv/issues'

def scmUrl = 'https://github.com/deephaven/deephaven-csv'
def scmConnection = 'scm:git:git://github.com/deephaven/deephaven-csv.git'
def scmDevConnection = 'scm:git:ssh://github.com/deephaven/deephaven-csv.git'

publishing {
publications {
mavenJava(MavenPublication) {
from components.java
artifactId = archivesBaseName
pom {
name = archivesBaseName
description = project.description
url = projectUrl
organization {
name = orgName
url = orgUrl
}
licenses {
license {
name = licenseName
url = licenseUrl
}
}
scm {
url = scmUrl
connection = scmConnection
developerConnection = scmDevConnection
}
issueManagement {
system = issuesSystem
url = issuesUrl
}
developers {
developer {
id = developerId
name = developerName
email = developerEmail
organization = orgName
organizationUrl = orgUrl
}
}
}
}
}
repositories {
maven {
name = 'ossrh'
// ossrhUsername, ossrhPassword
credentials(PasswordCredentials)
def releasesRepoUrl = 'https://s01.oss.sonatype.org/service/local/staging/deploy/maven2/'
def snapshotsRepoUrl = 'https://s01.oss.sonatype.org/content/repositories/snapshots/'
url = version.endsWith('SNAPSHOT') ? snapshotsRepoUrl : releasesRepoUrl
}
}
}

signing {
sign publishing.publications.mavenJava
String signingKey = findProperty('signingKey')
String signingPassword = findProperty('signingPassword')
if (signingKey != null && signingPassword != null) {
// In CI, it's harder to pass a file; so if specified, we use the in-memory version.
useInMemoryPgpKeys(signingKey, signingPassword)
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@
* </ul>
*
* These dimensions are broken out in the following way:
* <ul>
* <li>Small byte "strings" are packed into a byte block, and we maintain a linked list of these byte blocks.
* <li>"Large" byte "strings" are stored directly, meaning a byte[] array is allocated for their data, then a reference
* to that array is added to a byte-array block. (And again, we maintain a linked list of these byte-array blocks). It
Expand Down
18 changes: 9 additions & 9 deletions src/main/java/io/deephaven/csv/densestorage/QueueReader.java
Original file line number Diff line number Diff line change
Expand Up @@ -36,28 +36,28 @@ protected QueueReader(Object sync, QueueNode<TARRAY> node) {
* CharReader:
*
* <pre>
* <code>
* {@code
* if (current + size > end) {
* if (!tryRefill(size)) {
* return false;
* }
* typedBlock = genericBlock;
* }
* </code>
* }
* </pre>
*
* The "if" in the caller is actually checking for multiple cases in a single comparison. One is a normal "buffer
* empty, needs to be refilled" case. The other is a bad "something went terribly wrong" case.
*
* <ul>
* <li>Case 1, The "buffer empty" case. Then current == end, and therefore current + size > end (assuming size > 0,
* which it always is). Therefore, the 'if' inside the tryGetXXX code would evaluate to true, so the tryGetXXX code
* would call this method. Then this method refills the buffer.
* <li>Case 1, The "buffer empty" case. Then {@code current == end}, and therefore {@code current + size > end}
* (assuming {@code size > 0}, which it always is). Therefore, the 'if' inside the tryGetXXX code would evaluate to
* true, so the tryGetXXX code would call this method. Then this method refills the buffer.
* <li>Case 2: The buffer is not empty, but A logic error (which can't happen if the code is correct) has caused the
* requested slice to go past the end of the block. Then current < end but current + size > end. Again, the 'if'
* inside the tryGetXXX code would evaluate to true, so the tryGetXXX code would call this method. But then the
* first line of our method detects the past-the-end condition and throws an exception.
* <li>Case 3: The "buffer can satisfy the request" case. Then current + size &lt;= end, so the 'if' inside the
* requested slice to go past the end of the block. Then {@code current < end} but {@code current + size > end}.
* Again, the 'if' inside the tryGetXXX code would evaluate to true, so the tryGetXXX code would call this method.
* But then the first line of our method detects the past-the-end condition and throws an exception.
* <li>Case 3: The "buffer can satisfy the request" case. Then {@code current + size <= end}, so the 'if' inside the
* tryGetXXX code would evaluate to false, and the tryGetXXX method doesn't call this method.
* </ul>
*/
Expand Down
2 changes: 2 additions & 0 deletions src/main/java/io/deephaven/csv/densestorage/QueueWriter.java
Original file line number Diff line number Diff line change
Expand Up @@ -138,12 +138,14 @@ private void flush(boolean isLast) {
* This method exists as a helper method for a subclass' addXXX method. A typical implementation is in CharWriter:
*
* <pre>
* {@code
* final int sliceSize = cs.size();
* final boolean flushHappened = current + sliceSize > end;
* if (flushHappened) {
* typedBlock = flushAndAllocate(sliceSize);
* }
* ...
* }
* </pre>
*
* The "flushHappened" variable (which at the point of its definition would be more precisely interpreted as "flush
Expand Down
10 changes: 6 additions & 4 deletions src/main/java/io/deephaven/csv/parsers/Parser.java
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,13 @@ public interface Parser<TARRAY> {
int CHUNK_SIZE = 65536 * 4;

/**
* Make a context object for the parser. Sample implementation: <code><pre>
* Make a context object for the parser. Sample implementation:
*
* <pre>
* final MySink sink = new MySink();
* return new ParserContext<>(sink, null, new MyType[chunkSize]);
* </pre></code>
*
* return new ParserContext&lt;&gt;(sink, null, new MyType[chunkSize]);
* </pre>
*
* <p>
* Note that parsers other than {Byte,Short,Int,Long}Parser can leave the source field null, as in the above
* example.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,9 @@ public final class ParseDenseStorageToColumn {
* @param parsers The set of parsers to try. If null, then {@link Parsers#DEFAULT} will be used.
* @param nullValueLiteral If a cell text is equal to this value, it will be interpreted as the null value.
* Typically set to the empty string.
* @param nullParser The Parser to use if parsers.size() > 1 but the column contains all null values. This is needed
* as a backstop because otherwise type inference would have no way to choose among the multiple parsers.
* @param nullParser The Parser to use if {@code parsers.size() > 1} but the column contains all null values. This
* is needed as a backstop because otherwise type inference would have no way to choose among the multiple
* parsers.
* @param sinkFactory Factory that makes all of the Sinks of various types, used to consume the data we produce.
* @return The {@link Sink}, provided by the caller's {@link SinkFactory}, that was selected to hold the column
* data.
Expand Down
8 changes: 6 additions & 2 deletions src/main/java/io/deephaven/csv/sinks/Sink.java
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,10 @@
*/
public interface Sink<TARRAY> {
/**
* Write a chunk of data to the target data structure. Sample implementation: <code><pre>
* Write a chunk of data to the target data structure. Sample implementation:
*
* <pre>
* {@code
* if (appending) {
* while (mycolumn.size() < destBegin) {
* myColumn.addNull();
Expand All @@ -36,7 +39,8 @@ public interface Sink<TARRAY> {
* }
*
* }
* </pre></code>
* }
* </pre>
*
* @param src The chunk of data, a typed array (short[], double[], etc) with valid elements in the half-open
* interval {@code [0..(destEnd - destBegin))}.
Expand Down
24 changes: 13 additions & 11 deletions src/main/java/io/deephaven/csv/sinks/Source.java
Original file line number Diff line number Diff line change
Expand Up @@ -11,18 +11,20 @@
*/
public interface Source<TARRAY> {
/**
* Read a chunk of data from the src data structure. Sample implementation: <code><pre>
* int destIndex = 0;
* for (long srcIndex = srcBegin; srcIndex != srcEnd; ++srcIndex) {
* if (myColumn.hasNullAt(srcIndex)) {
* isNull[destIndex] = true;
* } else {
* dest[destIndex] = myColumn.getValueAt(srcIndex);
* isNull[destIndex] = false;
* }
* ++destIndex;
* Read a chunk of data from the src data structure. Sample implementation:
*
* <pre>
* int destIndex = 0;
* for (long srcIndex = srcBegin; srcIndex != srcEnd; ++srcIndex) {
* if (myColumn.hasNullAt(srcIndex)) {
* isNull[destIndex] = true;
* } else {
* dest[destIndex] = myColumn.getValueAt(srcIndex);
* isNull[destIndex] = false;
* }
* </pre></code>
* ++destIndex;
* }
* </pre>
*
* @param dest The chunk of data used to store values copied from the caller's column data structure.
* @param isNull A boolean array, with the same range of valid elements. A "true" value at position {@code i} means
Expand Down