From fb4d596eaf949eb6716b39771a8cf6267b531f7c Mon Sep 17 00:00:00 2001 From: Zach Kimberg Date: Tue, 10 May 2022 14:17:15 -0700 Subject: [PATCH] [0.17.0] Increase Final Version for release branch (#1628) --- android/README.md | 8 +++--- api/README.md | 2 +- basicdataset/README.md | 2 +- bom/README.md | 6 ++-- djl-zero/README.md | 2 +- docs/development/development_guideline.md | 2 +- docs/development/how_to_use_dataset.md | 4 +-- docs/development/memory_management.md | 2 +- docs/development/troubleshooting.md | 2 +- docs/hybrid_engine.md | 6 ++-- docs/load_model.md | 2 +- engines/dlr/dlr-engine/README.md | 4 +-- engines/ml/xgboost/README.md | 4 +-- engines/mxnet/mxnet-engine/README.md | 4 +-- engines/mxnet/mxnet-model-zoo/README.md | 2 +- .../onnxruntime/onnxruntime-engine/README.md | 8 +++--- .../paddlepaddle-model-zoo/README.md | 2 +- engines/pytorch/pytorch-engine/README.md | 4 +-- engines/pytorch/pytorch-model-zoo/README.md | 2 +- engines/tensorflow/tensorflow-api/README.md | 2 +- .../tensorflow/tensorflow-engine/README.md | 4 +-- .../tensorflow/tensorflow-model-zoo/README.md | 2 +- engines/tensorrt/README.md | 4 +-- engines/tflite/tflite-engine/README.md | 4 +-- extensions/aws-ai/README.md | 2 +- extensions/benchmark/README.md | 16 +++++------ extensions/benchmark/snapcraft/snapcraft.yaml | 2 +- extensions/fasttext/README.md | 2 +- extensions/hadoop/README.md | 2 +- extensions/opencv/README.md | 2 +- extensions/sentencepiece/README.md | 2 +- extensions/tokenizers/README.md | 2 +- jupyter/BERTQA.ipynb | 10 +++---- jupyter/load_mxnet_model.ipynb | 8 +++--- jupyter/load_pytorch_model.ipynb | 6 ++-- jupyter/mxnet/load_your_own_mxnet_bert.ipynb | 10 +++---- jupyter/object_detection_with_model_zoo.ipynb | 6 ++-- .../machine_learning_with_ONNXRuntime.ipynb | 4 +-- .../face_mask_detection_paddlepaddle.ipynb | 6 ++-- .../face_mask_detection_paddlepaddle_zh.ipynb | 6 ++-- jupyter/paddlepaddle/paddle_ocr_java.ipynb | 6 ++-- jupyter/paddlepaddle/paddle_ocr_java_zh.ipynb | 6 ++-- .../pytorch/load_your_own_pytorch_bert.ipynb | 10 +++---- ...fication_using_BERT_on_Amazon_Review.ipynb | 8 +++--- jupyter/tensorflow/pneumonia_detection.ipynb | 8 +++--- ...fication_using_BERT_on_Amazon_Review.ipynb | 6 ++-- .../inference_with_tensorflow_lite.ipynb | 8 +++--- jupyter/transfer_learning_on_cifar10.ipynb | 8 +++--- .../01_create_your_first_network.ipynb | 14 +++++----- .../tutorial/02_train_your_first_model.ipynb | 28 +++++++++---------- ...image_classification_with_your_model.ipynb | 14 +++++----- model-zoo/README.md | 2 +- 52 files changed, 144 insertions(+), 144 deletions(-) diff --git a/android/README.md b/android/README.md index 79047dd5fcd..596278f7f53 100644 --- a/android/README.md +++ b/android/README.md @@ -15,9 +15,9 @@ In gradle, you can include the snapshot repository and add the 4 modules in your ``` dependencies { - implementation "ai.djl:api:0.16.0-SNAPSHOT" - implementation "ai.djl.android:core:0.16.0-SNAPSHOT" - androidRuntimeOnly "ai.djl.pytorch:pytorch-engine:0.16.0-SNAPSHOT" - androidRuntimeOnly "ai.djl.android:pytorch-native:0.16.0-SNAPSHOT" + implementation "ai.djl:api:0.17.0-SNAPSHOT" + implementation "ai.djl.android:core:0.17.0-SNAPSHOT" + androidRuntimeOnly "ai.djl.pytorch:pytorch-engine:0.17.0-SNAPSHOT" + androidRuntimeOnly "ai.djl.android:pytorch-native:0.17.0-SNAPSHOT" } ``` diff --git a/api/README.md b/api/README.md index 85ac02b1c4c..918759fa8f0 100644 --- a/api/README.md +++ b/api/README.md @@ -35,7 +35,7 @@ You can pull the DJL API from the central Maven repository by including the foll ai.djl api - 0.16.0 + 0.17.0 ``` diff --git a/basicdataset/README.md b/basicdataset/README.md index b291e9439a7..c342cd15d43 100644 --- a/basicdataset/README.md +++ b/basicdataset/README.md @@ -39,7 +39,7 @@ You can pull the module from the central Maven repository by including the follo ai.djl basicdataset - 0.16.0 + 0.17.0 ``` diff --git a/bom/README.md b/bom/README.md index b9ead760c2e..5d6cf821b2a 100644 --- a/bom/README.md +++ b/bom/README.md @@ -22,7 +22,7 @@ will need to mention the type as pom and the scope as import) as the following: ai.djl bom - 0.16.0 + 0.17.0 pom import @@ -38,7 +38,7 @@ will need to mention the type as pom and the scope as import) as the following: ai.djl bom - 0.16.0 + 0.17.0 pom import @@ -70,7 +70,7 @@ will need to mention the type as pom and the scope as import) as the following: - First you need add BOM into your build.gradle file as the following: ``` - implementation platform("ai.djl:bom:0.16.0") + implementation platform("ai.djl:bom:0.17.0") ``` - Then you import the desired DJL modules into to you pom.xml file (no version is needed): diff --git a/djl-zero/README.md b/djl-zero/README.md index 05cf15b26f0..2b8615a127a 100644 --- a/djl-zero/README.md +++ b/djl-zero/README.md @@ -34,6 +34,6 @@ You can pull the module from the central Maven repository by including the follo ai.djl djl-zero - 0.16.0 + 0.17.0 ``` diff --git a/docs/development/development_guideline.md b/docs/development/development_guideline.md index aed0c95f0aa..41e7255566f 100644 --- a/docs/development/development_guideline.md +++ b/docs/development/development_guideline.md @@ -159,7 +159,7 @@ You can create your own NDArray renderer as follows: Please make sure to: - Check the "On-demand" option, which causes IntelliJ to only render the NDArray when you click on the variable. -- Change the "Use following expression" field to something like [toDebugString(100, 10, 10, 20)](https://javadoc.io/static/ai.djl/api/0.16.0/ai/djl/ndarray/NDArray.html#toDebugString-int-int-int-int-) +- Change the "Use following expression" field to something like [toDebugString(100, 10, 10, 20)](https://javadoc.io/static/ai.djl/api/0.17.0/ai/djl/ndarray/NDArray.html#toDebugString-int-int-int-int-) if you want to adjust the range of NDArray's debug output. ## Common Problems diff --git a/docs/development/how_to_use_dataset.md b/docs/development/how_to_use_dataset.md index 93c31c891af..206957b04a8 100644 --- a/docs/development/how_to_use_dataset.md +++ b/docs/development/how_to_use_dataset.md @@ -108,8 +108,8 @@ api group: 'org.apache.commons', name: 'commons-csv', version: '1.7' In order to extend the dataset, the following dependencies are required: ``` -api "ai.djl:api:0.16.0" -api "ai.djl:basicdataset:0.16.0" +api "ai.djl:api:0.17.0" +api "ai.djl:basicdataset:0.17.0" ``` There are four parts we need to implement for CSVDataset. diff --git a/docs/development/memory_management.md b/docs/development/memory_management.md index 1e39862d78b..2c5d8551c85 100644 --- a/docs/development/memory_management.md +++ b/docs/development/memory_management.md @@ -37,7 +37,7 @@ The intermediate NDArrays involving in training case are usually In general, all the parameters in the model should be associated with Model level NDManager. All of the input and output NDArrays should be associated with one NDManager which is one level down to the model NDManager. -Please check if you call [batch.close()](https://javadoc.io/static/ai.djl/api/0.16.0/ai/djl/training/dataset/Batch.html#close--) +Please check if you call [batch.close()](https://javadoc.io/static/ai.djl/api/0.17.0/ai/djl/training/dataset/Batch.html#close--) to release one batch of the dataset at the end of each batch. If you still see the memory grows as the training process goes, it is most likely that intermediate NDArrays are attached to the Model(Block) parameter level. As a result, those NDArrays would not closed until the training is finished. diff --git a/docs/development/troubleshooting.md b/docs/development/troubleshooting.md index e8a0aa6d5db..c884bed9bbc 100644 --- a/docs/development/troubleshooting.md +++ b/docs/development/troubleshooting.md @@ -28,7 +28,7 @@ For example, adding MXNet engine dependencies: Gradle: ``` -implementation "ai.djl.mxnet:mxnet-engine:0.16.0" +implementation "ai.djl.mxnet:mxnet-engine:0.17.0" // See https://github.com/deepjavalibrary/djl/blob/master/engines/mxnet/mxnet-engine/README.md for more MXNet library selection options runtimeOnly "ai.djl.mxnet:mxnet-native-auto:1.8.0" ``` diff --git a/docs/hybrid_engine.md b/docs/hybrid_engine.md index 31348eb81f9..374ad17ea2f 100644 --- a/docs/hybrid_engine.md +++ b/docs/hybrid_engine.md @@ -22,17 +22,17 @@ to run in a hybrid mode: To use it along with Apache MXNet for additional API support, add the following two dependencies: ``` -runtimeOnly "ai.djl.mxnet:mxnet-engine:0.16.0" +runtimeOnly "ai.djl.mxnet:mxnet-engine:0.17.0" ``` You can also use PyTorch or TensorFlow Engine as the supplemental engine by adding their corresponding dependencies. ``` -runtimeOnly "ai.djl.pytorch:pytorch-engine:0.16.0" +runtimeOnly "ai.djl.pytorch:pytorch-engine:0.17.0" ``` ``` -runtimeOnly "ai.djl.tensorflow:tensorflow-engine:0.16.0" +runtimeOnly "ai.djl.tensorflow:tensorflow-engine:0.17.0" ``` ## How Hybrid works diff --git a/docs/load_model.md b/docs/load_model.md index c7ac1ae5259..b4a03ff6ca3 100644 --- a/docs/load_model.md +++ b/docs/load_model.md @@ -181,7 +181,7 @@ Here is a few tips you can use to help you debug model loading issue: See [here](development/configure_logging.md#configure-logging-level) for how to enable debug log #### List models programmatically in your code -You can use [ModelZoo.listModels()](https://javadoc.io/static/ai.djl/api/0.16.0/ai/djl/repository/zoo/ModelZoo.html#listModels--) API to query available models. +You can use [ModelZoo.listModels()](https://javadoc.io/static/ai.djl/api/0.17.0/ai/djl/repository/zoo/ModelZoo.html#listModels--) API to query available models. #### List available models using DJL command line diff --git a/engines/dlr/dlr-engine/README.md b/engines/dlr/dlr-engine/README.md index b16a4f5ee97..ea7596c43dd 100644 --- a/engines/dlr/dlr-engine/README.md +++ b/engines/dlr/dlr-engine/README.md @@ -29,13 +29,13 @@ The javadocs output is generated in the `build/doc/javadoc` folder. ## Installation You can pull the DLR engine from the central Maven repository by including the following dependency: -- ai.djl.dlr:dlr-engine:0.16.0 +- ai.djl.dlr:dlr-engine:0.17.0 ```xml ai.djl.dlr dlr-engine - 0.16.0 + 0.17.0 runtime ``` diff --git a/engines/ml/xgboost/README.md b/engines/ml/xgboost/README.md index 7319b9832dd..5031e90bd02 100644 --- a/engines/ml/xgboost/README.md +++ b/engines/ml/xgboost/README.md @@ -37,13 +37,13 @@ XGBoost can only run on top of the Linux/Mac machine. User can build from source ## Installation You can pull the XGBoost engine from the central Maven repository by including the following dependency: -- ai.djl.ml.xgboost:xgboost:0.16.0 +- ai.djl.ml.xgboost:xgboost:0.17.0 ```xml ai.djl.ml.xgboost xgboost - 0.16.0 + 0.17.0 runtime ``` diff --git a/engines/mxnet/mxnet-engine/README.md b/engines/mxnet/mxnet-engine/README.md index 1b9ad35e570..db1788c6c26 100644 --- a/engines/mxnet/mxnet-engine/README.md +++ b/engines/mxnet/mxnet-engine/README.md @@ -7,7 +7,7 @@ This module contains the Deep Java Library (DJL) EngineProvider for Apache MXNet We don't recommend that developers use classes in this module directly. Use of these classes will couple your code with Apache MXNet and make switching between engines difficult. Even so, developers are not restricted from using engine-specific features. For more information, -see [NDManager#invoke()](https://javadoc.io/static/ai.djl/api/0.16.0/ai/djl/ndarray/NDManager.html#invoke-java.lang.String-ai.djl.ndarray.NDArray:A-ai.djl.ndarray.NDArray:A-ai.djl.util.PairList-). +see [NDManager#invoke()](https://javadoc.io/static/ai.djl/api/0.17.0/ai/djl/ndarray/NDManager.html#invoke-java.lang.String-ai.djl.ndarray.NDArray:A-ai.djl.ndarray.NDArray:A-ai.djl.util.PairList-). ## Documentation @@ -32,7 +32,7 @@ You can pull the MXNet engine from the central Maven repository by including the ai.djl.mxnet mxnet-engine - 0.16.0 + 0.17.0 runtime ``` diff --git a/engines/mxnet/mxnet-model-zoo/README.md b/engines/mxnet/mxnet-model-zoo/README.md index 9f7713a1cb4..f98d79b8095 100644 --- a/engines/mxnet/mxnet-model-zoo/README.md +++ b/engines/mxnet/mxnet-model-zoo/README.md @@ -27,7 +27,7 @@ You can pull the MXNet engine from the central Maven repository by including the ai.djl.mxnet mxnet-model-zoo - 0.16.0 + 0.17.0 ``` diff --git a/engines/onnxruntime/onnxruntime-engine/README.md b/engines/onnxruntime/onnxruntime-engine/README.md index 900f987c912..ffcdad1c319 100644 --- a/engines/onnxruntime/onnxruntime-engine/README.md +++ b/engines/onnxruntime/onnxruntime-engine/README.md @@ -37,13 +37,13 @@ for the official ONNX Runtime project. ## Installation You can pull the ONNX Runtime engine from the central Maven repository by including the following dependency: -- ai.djl.onnxruntime:onnxruntime-engine:0.16.0 +- ai.djl.onnxruntime:onnxruntime-engine:0.17.0 ```xml ai.djl.onnxruntime onnxruntime-engine - 0.16.0 + 0.17.0 runtime ``` @@ -61,7 +61,7 @@ Maven: ai.djl.onnxruntime onnxruntime-engine - 0.16.0 + 0.17.0 runtime @@ -80,7 +80,7 @@ Maven: Gradle: ``` - implementation("ai.djl.onnxruntime:onnxruntime-engine:0.16.0") { + implementation("ai.djl.onnxruntime:onnxruntime-engine:0.17.0") { exclude group: "com.microsoft.onnxruntime", module: "onnxruntime" } implementation "com.microsoft.onnxruntime:onnxruntime_gpu:1.11.0" diff --git a/engines/paddlepaddle/paddlepaddle-model-zoo/README.md b/engines/paddlepaddle/paddlepaddle-model-zoo/README.md index 98adc3d8f9f..ab5a1ac8a9d 100644 --- a/engines/paddlepaddle/paddlepaddle-model-zoo/README.md +++ b/engines/paddlepaddle/paddlepaddle-model-zoo/README.md @@ -26,7 +26,7 @@ from the central Maven repository by including the following dependency: ai.djl.paddlepaddle paddlepaddle-model-zoo - 0.16.0 + 0.17.0 ``` diff --git a/engines/pytorch/pytorch-engine/README.md b/engines/pytorch/pytorch-engine/README.md index 48d163943bb..0563e296833 100644 --- a/engines/pytorch/pytorch-engine/README.md +++ b/engines/pytorch/pytorch-engine/README.md @@ -24,13 +24,13 @@ The javadocs output is built in the `build/doc/javadoc` folder. ## Installation You can pull the PyTorch engine from the central Maven repository by including the following dependency: -- ai.djl.pytorch:pytorch-engine:0.16.0 +- ai.djl.pytorch:pytorch-engine:0.17.0 ```xml ai.djl.pytorch pytorch-engine - 0.16.0 + 0.17.0 runtime ``` diff --git a/engines/pytorch/pytorch-model-zoo/README.md b/engines/pytorch/pytorch-model-zoo/README.md index 0f77dd3a5e9..fd9a973c16d 100644 --- a/engines/pytorch/pytorch-model-zoo/README.md +++ b/engines/pytorch/pytorch-model-zoo/README.md @@ -25,7 +25,7 @@ You can pull the PyTorch engine from the central Maven repository by including t ai.djl.pytorch pytorch-model-zoo - 0.16.0 + 0.17.0 ``` diff --git a/engines/tensorflow/tensorflow-api/README.md b/engines/tensorflow/tensorflow-api/README.md index 3bca85f54b0..343a0271add 100644 --- a/engines/tensorflow/tensorflow-api/README.md +++ b/engines/tensorflow/tensorflow-api/README.md @@ -16,6 +16,6 @@ You can pull the TensorFlow core java API from the central Maven repository by i ai.djl.tensorflow tensorflow-api - 0.16.0 + 0.17.0 ``` diff --git a/engines/tensorflow/tensorflow-engine/README.md b/engines/tensorflow/tensorflow-engine/README.md index 840a3e35db5..3ab09d45a24 100644 --- a/engines/tensorflow/tensorflow-engine/README.md +++ b/engines/tensorflow/tensorflow-engine/README.md @@ -28,13 +28,13 @@ The javadocs output is built in the `build/doc/javadoc` folder. You can pull the TensorFlow engine from the central Maven repository by including the following dependency: -- ai.djl.tensorflow:tensorflow-engine:0.16.0 +- ai.djl.tensorflow:tensorflow-engine:0.17.0 ```xml ai.djl.tensorflow tensorflow-engine - 0.16.0 + 0.17.0 runtime ``` diff --git a/engines/tensorflow/tensorflow-model-zoo/README.md b/engines/tensorflow/tensorflow-model-zoo/README.md index a949aa87c8e..881997de59e 100644 --- a/engines/tensorflow/tensorflow-model-zoo/README.md +++ b/engines/tensorflow/tensorflow-model-zoo/README.md @@ -26,7 +26,7 @@ from the central Maven repository by including the following dependency: ai.djl.tensorflow tensorflow-model-zoo - 0.16.0 + 0.17.0 ``` diff --git a/engines/tensorrt/README.md b/engines/tensorrt/README.md index 40a68f88542..744fe59891c 100644 --- a/engines/tensorrt/README.md +++ b/engines/tensorrt/README.md @@ -28,13 +28,13 @@ The javadocs output is generated in the `build/doc/javadoc` folder. ## Installation You can pull the TensorRT engine from the central Maven repository by including the following dependency: -- ai.djl.tensorrt:tensorrt:0.16.0 +- ai.djl.tensorrt:tensorrt:0.17.0 ```xml ai.djl.tensorrt tensorrt - 0.16.0 + 0.17.0 runtime ``` diff --git a/engines/tflite/tflite-engine/README.md b/engines/tflite/tflite-engine/README.md index a9de9d350d4..54bf582175f 100644 --- a/engines/tflite/tflite-engine/README.md +++ b/engines/tflite/tflite-engine/README.md @@ -24,13 +24,13 @@ The javadocs output is built in the `build/doc/javadoc` folder. ## Installation You can pull the TensorFlow Lite engine from the central Maven repository by including the following dependency: -- ai.djl.tflite:tflite-engine:0.16.0 +- ai.djl.tflite:tflite-engine:0.17.0 ```xml ai.djl.tflite tflite-engine - 0.16.0 + 0.17.0 runtime ``` diff --git a/extensions/aws-ai/README.md b/extensions/aws-ai/README.md index 3493c684885..ca57eefa688 100644 --- a/extensions/aws-ai/README.md +++ b/extensions/aws-ai/README.md @@ -58,6 +58,6 @@ You can pull the module from the central Maven repository by including the follo ai.djl.aws aws-ai - 0.16.0 + 0.17.0 ``` diff --git a/extensions/benchmark/README.md b/extensions/benchmark/README.md index 4fcb70a0fc5..56ec1b78145 100644 --- a/extensions/benchmark/README.md +++ b/extensions/benchmark/README.md @@ -48,25 +48,25 @@ sudo snap alias djlbench djl-bench - Or download .deb package from S3 ``` -curl -O https://publish.djl.ai/djl-bench/0.16.0/djl-bench_0.16.0-1_all.deb -sudo dpkg -i djl-bench_0.16.0-1_all.deb +curl -O https://publish.djl.ai/djl-bench/0.17.0/djl-bench_0.17.0-1_all.deb +sudo dpkg -i djl-bench_0.17.0-1_all.deb ``` For centOS or Amazon Linux 2 -You can download djl-bench zip file from [here](https://publish.djl.ai/djl-bench/0.16.0/benchmark-0.16.0.zip). +You can download djl-bench zip file from [here](https://publish.djl.ai/djl-bench/0.17.0/benchmark-0.17.0.zip). ``` -curl -O https://publish.djl.ai/djl-bench/0.16.0/benchmark-0.16.0.zip -unzip benchmark-0.16.0.zip -rm benchmark-0.16.0.zip -sudo ln -s $PWD/benchmark-0.16.0/bin/benchmark /usr/bin/djl-bench +curl -O https://publish.djl.ai/djl-bench/0.17.0/benchmark-0.17.0.zip +unzip benchmark-0.17.0.zip +rm benchmark-0.17.0.zip +sudo ln -s $PWD/benchmark-0.17.0/bin/benchmark /usr/bin/djl-bench ``` For Windows We are considering to create a `chocolatey` package for Windows. For the time being, you can -download djl-bench zip file from [here](https://publish.djl.ai/djl-bench/0.16.0/benchmark-0.16.0.zip). +download djl-bench zip file from [here](https://publish.djl.ai/djl-bench/0.17.0/benchmark-0.17.0.zip). Or you can run benchmark using gradle: diff --git a/extensions/benchmark/snapcraft/snapcraft.yaml b/extensions/benchmark/snapcraft/snapcraft.yaml index b4b6a691d68..afa85c747db 100644 --- a/extensions/benchmark/snapcraft/snapcraft.yaml +++ b/extensions/benchmark/snapcraft/snapcraft.yaml @@ -1,5 +1,5 @@ name: djlbench -version: '0.16.0' +version: '0.17.0' title: DJL Benhmark license: Apache-2.0 summary: A machine learning benchmarking toolkit diff --git a/extensions/fasttext/README.md b/extensions/fasttext/README.md index 9063c64b5c3..0aba6a866cd 100644 --- a/extensions/fasttext/README.md +++ b/extensions/fasttext/README.md @@ -34,7 +34,7 @@ You can pull the fastText engine from the central Maven repository by including ai.djl.fasttext fasttext-engine - 0.16.0 + 0.17.0 ``` diff --git a/extensions/hadoop/README.md b/extensions/hadoop/README.md index f5a56268433..cf117981eec 100644 --- a/extensions/hadoop/README.md +++ b/extensions/hadoop/README.md @@ -52,6 +52,6 @@ You can pull the module from the central Maven repository by including the follo ai.djl.hadoop hadoop - 0.16.0 + 0.17.0 ``` diff --git a/extensions/opencv/README.md b/extensions/opencv/README.md index f296bd0bacf..62d47f1333e 100644 --- a/extensions/opencv/README.md +++ b/extensions/opencv/README.md @@ -21,6 +21,6 @@ You can pull the module from the central Maven repository by including the follo ai.djl.opencv opencv - 0.16.0 + 0.17.0 ``` diff --git a/extensions/sentencepiece/README.md b/extensions/sentencepiece/README.md index 099864cb487..d850b336d1c 100644 --- a/extensions/sentencepiece/README.md +++ b/extensions/sentencepiece/README.md @@ -21,6 +21,6 @@ You can pull the module from the central Maven repository by including the follo ai.djl.sentencepiece sentencepiece - 0.16.0 + 0.17.0 ``` diff --git a/extensions/tokenizers/README.md b/extensions/tokenizers/README.md index 33846eaca66..256f5741299 100644 --- a/extensions/tokenizers/README.md +++ b/extensions/tokenizers/README.md @@ -21,6 +21,6 @@ You can pull the module from the central Maven repository by including the follo ai.djl.huggingface tokenizers - 0.16.0 + 0.17.0 ``` diff --git a/jupyter/BERTQA.ipynb b/jupyter/BERTQA.ipynb index 7b7dcd9dbb8..4eedc4190b4 100644 --- a/jupyter/BERTQA.ipynb +++ b/jupyter/BERTQA.ipynb @@ -46,11 +46,11 @@ "source": [ "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n", "\n", - "%maven ai.djl:api:0.16.0\n", - "%maven ai.djl.mxnet:mxnet-engine:0.16.0\n", - "%maven ai.djl.mxnet:mxnet-model-zoo:0.16.0\n", - "%maven ai.djl.pytorch:pytorch-engine:0.16.0\n", - "%maven ai.djl.pytorch:pytorch-model-zoo:0.16.0\n", + "%maven ai.djl:api:0.17.0\n", + "%maven ai.djl.mxnet:mxnet-engine:0.17.0\n", + "%maven ai.djl.mxnet:mxnet-model-zoo:0.17.0\n", + "%maven ai.djl.pytorch:pytorch-engine:0.17.0\n", + "%maven ai.djl.pytorch:pytorch-model-zoo:0.17.0\n", "%maven org.slf4j:slf4j-simple:1.7.32" ] }, diff --git a/jupyter/load_mxnet_model.ipynb b/jupyter/load_mxnet_model.ipynb index 4a73ee6e915..4031a926c3f 100644 --- a/jupyter/load_mxnet_model.ipynb +++ b/jupyter/load_mxnet_model.ipynb @@ -22,10 +22,10 @@ "source": [ "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n", "\n", - "%maven ai.djl:api:0.16.0\n", - "%maven ai.djl:model-zoo:0.16.0\n", - "%maven ai.djl.mxnet:mxnet-engine:0.16.0\n", - "%maven ai.djl.mxnet:mxnet-model-zoo:0.16.0\n", + "%maven ai.djl:api:0.17.0\n", + "%maven ai.djl:model-zoo:0.17.0\n", + "%maven ai.djl.mxnet:mxnet-engine:0.17.0\n", + "%maven ai.djl.mxnet:mxnet-model-zoo:0.17.0\n", "%maven org.slf4j:slf4j-simple:1.7.32" ] }, diff --git a/jupyter/load_pytorch_model.ipynb b/jupyter/load_pytorch_model.ipynb index 7fbe6df1e26..13ceb96fbcb 100644 --- a/jupyter/load_pytorch_model.ipynb +++ b/jupyter/load_pytorch_model.ipynb @@ -25,8 +25,8 @@ "source": [ "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n", "\n", - "%maven ai.djl:api:0.16.0\n", - "%maven ai.djl.pytorch:pytorch-engine:0.16.0\n", + "%maven ai.djl:api:0.17.0\n", + "%maven ai.djl.pytorch:pytorch-engine:0.17.0\n", "%maven org.slf4j:slf4j-simple:1.7.32" ] }, @@ -106,7 +106,7 @@ "...\n", "```\n", "\n", - "Then, we will use this pipeline to create the [`Translator`](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/translate/Translator.html)" + "Then, we will use this pipeline to create the [`Translator`](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/translate/Translator.html)" ] }, { diff --git a/jupyter/mxnet/load_your_own_mxnet_bert.ipynb b/jupyter/mxnet/load_your_own_mxnet_bert.ipynb index 2031e18bd0b..338b7446f45 100644 --- a/jupyter/mxnet/load_your_own_mxnet_bert.ipynb +++ b/jupyter/mxnet/load_your_own_mxnet_bert.ipynb @@ -41,9 +41,9 @@ "source": [ "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n", "\n", - "%maven ai.djl:api:0.16.0\n", - "%maven ai.djl.mxnet:mxnet-engine:0.16.0\n", - "%maven ai.djl.mxnet:mxnet-model-zoo:0.16.0\n", + "%maven ai.djl:api:0.17.0\n", + "%maven ai.djl.mxnet:mxnet-engine:0.17.0\n", + "%maven ai.djl.mxnet:mxnet-model-zoo:0.17.0\n", "%maven org.slf4j:slf4j-simple:1.7.32" ] }, @@ -310,7 +310,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Armed with the needed knowledge, you can write an implementation of the `Translator` interface. `BertTranslator` uses the code snippets explained previously to implement the `processInput`method. For more information, see [`NDManager`](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/ndarray/NDManager.html).\n", + "Armed with the needed knowledge, you can write an implementation of the `Translator` interface. `BertTranslator` uses the code snippets explained previously to implement the `processInput`method. For more information, see [`NDManager`](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/ndarray/NDManager.html).\n", "\n", "```\n", "manager.create(Number[] data, Shape)\n", @@ -389,7 +389,7 @@ "source": [ "Congrats! You have created your first Translator! We have pre-filled the `processOutput()` function to process the `NDList` and return it in a desired format. `processInput()` and `processOutput()` offer the flexibility to get the predictions from the model in any format you desire. \n", "\n", - "With the Translator implemented, you need to bring up the predictor that uses your `Translator` to start making predictions. You can find the usage for `Predictor` in the [Predictor Javadoc](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/inference/Predictor.html). Create a translator and use the `question` and `resourceDocument` provided previously." + "With the Translator implemented, you need to bring up the predictor that uses your `Translator` to start making predictions. You can find the usage for `Predictor` in the [Predictor Javadoc](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/inference/Predictor.html). Create a translator and use the `question` and `resourceDocument` provided previously." ] }, { diff --git a/jupyter/object_detection_with_model_zoo.ipynb b/jupyter/object_detection_with_model_zoo.ipynb index 6e0b718c694..5cd6611156f 100644 --- a/jupyter/object_detection_with_model_zoo.ipynb +++ b/jupyter/object_detection_with_model_zoo.ipynb @@ -21,9 +21,9 @@ "source": [ "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n", "\n", - "%maven ai.djl:api:0.16.0\n", - "%maven ai.djl.mxnet:mxnet-engine:0.16.0\n", - "%maven ai.djl.mxnet:mxnet-model-zoo:0.16.0\n", + "%maven ai.djl:api:0.17.0\n", + "%maven ai.djl.mxnet:mxnet-engine:0.17.0\n", + "%maven ai.djl.mxnet:mxnet-model-zoo:0.17.0\n", "%maven org.slf4j:slf4j-simple:1.7.32" ] }, diff --git a/jupyter/onnxruntime/machine_learning_with_ONNXRuntime.ipynb b/jupyter/onnxruntime/machine_learning_with_ONNXRuntime.ipynb index 9e77ac6fa4f..73611c2c896 100644 --- a/jupyter/onnxruntime/machine_learning_with_ONNXRuntime.ipynb +++ b/jupyter/onnxruntime/machine_learning_with_ONNXRuntime.ipynb @@ -54,8 +54,8 @@ "source": [ "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n", "\n", - "%maven ai.djl:api:0.16.0\n", - "%maven ai.djl.onnxruntime:onnxruntime-engine:0.16.0\n", + "%maven ai.djl:api:0.17.0\n", + "%maven ai.djl.onnxruntime:onnxruntime-engine:0.17.0\n", "%maven org.slf4j:slf4j-simple:1.7.32" ] }, diff --git a/jupyter/paddlepaddle/face_mask_detection_paddlepaddle.ipynb b/jupyter/paddlepaddle/face_mask_detection_paddlepaddle.ipynb index a5c778d9c7a..f4a55b009e4 100644 --- a/jupyter/paddlepaddle/face_mask_detection_paddlepaddle.ipynb +++ b/jupyter/paddlepaddle/face_mask_detection_paddlepaddle.ipynb @@ -26,12 +26,12 @@ "source": [ "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n", "\n", - "%maven ai.djl:api:0.16.0\n", - "%maven ai.djl.paddlepaddle:paddlepaddle-model-zoo:0.16.0\n", + "%maven ai.djl:api:0.17.0\n", + "%maven ai.djl.paddlepaddle:paddlepaddle-model-zoo:0.17.0\n", "%maven org.slf4j:slf4j-simple:1.7.32\n", "\n", "// second engine to do preprocessing and postprocessing\n", - "%maven ai.djl.pytorch:pytorch-engine:0.16.0" + "%maven ai.djl.pytorch:pytorch-engine:0.17.0" ] }, { diff --git a/jupyter/paddlepaddle/face_mask_detection_paddlepaddle_zh.ipynb b/jupyter/paddlepaddle/face_mask_detection_paddlepaddle_zh.ipynb index 4afd631e7d3..4a1271caef0 100644 --- a/jupyter/paddlepaddle/face_mask_detection_paddlepaddle_zh.ipynb +++ b/jupyter/paddlepaddle/face_mask_detection_paddlepaddle_zh.ipynb @@ -24,12 +24,12 @@ "source": [ "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n", "\n", - "%maven ai.djl:api:0.16.0\n", - "%maven ai.djl.paddlepaddle:paddlepaddle-model-zoo:0.16.0\n", + "%maven ai.djl:api:0.17.0\n", + "%maven ai.djl.paddlepaddle:paddlepaddle-model-zoo:0.17.0\n", "%maven org.slf4j:slf4j-simple:1.7.32\n", "\n", "// second engine to do preprocessing and postprocessing\n", - "%maven ai.djl.pytorch:pytorch-engine:0.16.0" + "%maven ai.djl.pytorch:pytorch-engine:0.17.0" ] }, { diff --git a/jupyter/paddlepaddle/paddle_ocr_java.ipynb b/jupyter/paddlepaddle/paddle_ocr_java.ipynb index c75842eb6bb..7836ba87602 100644 --- a/jupyter/paddlepaddle/paddle_ocr_java.ipynb +++ b/jupyter/paddlepaddle/paddle_ocr_java.ipynb @@ -25,12 +25,12 @@ "source": [ "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n", "\n", - "%maven ai.djl:api:0.16.0\n", - "%maven ai.djl.paddlepaddle:paddlepaddle-model-zoo:0.16.0\n", + "%maven ai.djl:api:0.17.0\n", + "%maven ai.djl.paddlepaddle:paddlepaddle-model-zoo:0.17.0\n", "%maven org.slf4j:slf4j-simple:1.7.32\n", "\n", "// second engine to do preprocessing and postprocessing\n", - "%maven ai.djl.pytorch:pytorch-engine:0.16.0" + "%maven ai.djl.pytorch:pytorch-engine:0.17.0" ] }, { diff --git a/jupyter/paddlepaddle/paddle_ocr_java_zh.ipynb b/jupyter/paddlepaddle/paddle_ocr_java_zh.ipynb index aca169821bc..bcb27e2e9a4 100644 --- a/jupyter/paddlepaddle/paddle_ocr_java_zh.ipynb +++ b/jupyter/paddlepaddle/paddle_ocr_java_zh.ipynb @@ -23,12 +23,12 @@ "source": [ "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n", "\n", - "%maven ai.djl:api:0.16.0\n", - "%maven ai.djl.paddlepaddle:paddlepaddle-model-zoo:0.16.0\n", + "%maven ai.djl:api:0.17.0\n", + "%maven ai.djl.paddlepaddle:paddlepaddle-model-zoo:0.17.0\n", "%maven org.slf4j:slf4j-simple:1.7.32\n", "\n", "// second engine to do preprocessing and postprocessing\n", - "%maven ai.djl.pytorch:pytorch-engine:0.16.0" + "%maven ai.djl.pytorch:pytorch-engine:0.17.0" ] }, { diff --git a/jupyter/pytorch/load_your_own_pytorch_bert.ipynb b/jupyter/pytorch/load_your_own_pytorch_bert.ipynb index 305a52b5d2b..9964662be5c 100644 --- a/jupyter/pytorch/load_your_own_pytorch_bert.ipynb +++ b/jupyter/pytorch/load_your_own_pytorch_bert.ipynb @@ -41,9 +41,9 @@ "source": [ "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n", "\n", - "%maven ai.djl:api:0.16.0\n", - "%maven ai.djl.pytorch:pytorch-engine:0.16.0\n", - "%maven ai.djl.pytorch:pytorch-model-zoo:0.16.0\n", + "%maven ai.djl:api:0.17.0\n", + "%maven ai.djl.pytorch:pytorch-engine:0.17.0\n", + "%maven ai.djl.pytorch:pytorch-model-zoo:0.17.0\n", "%maven org.slf4j:slf4j-simple:1.7.32" ] }, @@ -272,7 +272,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Armed with the needed knowledge, you can write an implementation of the `Translator` interface. `BertTranslator` uses the code snippets explained previously to implement the `processInput`method. For more information, see [`NDManager`](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/ndarray/NDManager.html).\n", + "Armed with the needed knowledge, you can write an implementation of the `Translator` interface. `BertTranslator` uses the code snippets explained previously to implement the `processInput`method. For more information, see [`NDManager`](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/ndarray/NDManager.html).\n", "\n", "```\n", "manager.create(Number[] data, Shape)\n", @@ -345,7 +345,7 @@ "source": [ "Congrats! You have created your first Translator! We have pre-filled the `processOutput()` function to process the `NDList` and return it in a desired format. `processInput()` and `processOutput()` offer the flexibility to get the predictions from the model in any format you desire. \n", "\n", - "With the Translator implemented, you need to bring up the predictor that uses your `Translator` to start making predictions. You can find the usage for `Predictor` in the [Predictor Javadoc](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/inference/Predictor.html). Create a translator and use the `question` and `resourceDocument` provided previously." + "With the Translator implemented, you need to bring up the predictor that uses your `Translator` to start making predictions. You can find the usage for `Predictor` in the [Predictor Javadoc](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/inference/Predictor.html). Create a translator and use the `question` and `resourceDocument` provided previously." ] }, { diff --git a/jupyter/rank_classification_using_BERT_on_Amazon_Review.ipynb b/jupyter/rank_classification_using_BERT_on_Amazon_Review.ipynb index b8d50006c51..24c1adf321b 100644 --- a/jupyter/rank_classification_using_BERT_on_Amazon_Review.ipynb +++ b/jupyter/rank_classification_using_BERT_on_Amazon_Review.ipynb @@ -38,13 +38,13 @@ "source": [ "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n", "\n", - "%maven ai.djl:api:0.16.0\n", - "%maven ai.djl:basicdataset:0.16.0\n", + "%maven ai.djl:api:0.17.0\n", + "%maven ai.djl:basicdataset:0.17.0\n", "%maven org.slf4j:slf4j-simple:1.7.32\n", - "%maven ai.djl.mxnet:mxnet-model-zoo:0.16.0\n", + "%maven ai.djl.mxnet:mxnet-model-zoo:0.17.0\n", "\n", "// PyTorch\n", - "// %maven ai.djl.pytorch:pytorch-model-zoo:0.16.0" + "// %maven ai.djl.pytorch:pytorch-model-zoo:0.17.0" ] }, { diff --git a/jupyter/tensorflow/pneumonia_detection.ipynb b/jupyter/tensorflow/pneumonia_detection.ipynb index d8bdbdb8ce4..97118bfeeb5 100644 --- a/jupyter/tensorflow/pneumonia_detection.ipynb +++ b/jupyter/tensorflow/pneumonia_detection.ipynb @@ -36,10 +36,10 @@ "source": [ "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n", "\n", - "%maven ai.djl:api:0.16.0\n", - "%maven ai.djl.tensorflow:tensorflow-api:0.16.0\n", - "%maven ai.djl.tensorflow:tensorflow-engine:0.16.0\n", - "%maven ai.djl.tensorflow:tensorflow-model-zoo:0.16.0\n", + "%maven ai.djl:api:0.17.0\n", + "%maven ai.djl.tensorflow:tensorflow-api:0.17.0\n", + "%maven ai.djl.tensorflow:tensorflow-engine:0.17.0\n", + "%maven ai.djl.tensorflow:tensorflow-model-zoo:0.17.0\n", "%maven org.slf4j:slf4j-simple:1.7.32" ] }, diff --git a/jupyter/tensorflow/rank_classification_using_BERT_on_Amazon_Review.ipynb b/jupyter/tensorflow/rank_classification_using_BERT_on_Amazon_Review.ipynb index e1fcfe60153..075b8e9e888 100644 --- a/jupyter/tensorflow/rank_classification_using_BERT_on_Amazon_Review.ipynb +++ b/jupyter/tensorflow/rank_classification_using_BERT_on_Amazon_Review.ipynb @@ -36,9 +36,9 @@ "source": [ "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n", "\n", - "%maven ai.djl:api:0.16.0\n", - "%maven ai.djl.tensorflow:tensorflow-engine:0.16.0\n", - "%maven ai.djl.tensorflow:tensorflow-api:0.16.0\n", + "%maven ai.djl:api:0.17.0\n", + "%maven ai.djl.tensorflow:tensorflow-engine:0.17.0\n", + "%maven ai.djl.tensorflow:tensorflow-api:0.17.0\n", "%maven org.slf4j:slf4j-simple:1.7.32" ] }, diff --git a/jupyter/tensorflow_lite/inference_with_tensorflow_lite.ipynb b/jupyter/tensorflow_lite/inference_with_tensorflow_lite.ipynb index c09fc0bf44a..e98c6f1f115 100644 --- a/jupyter/tensorflow_lite/inference_with_tensorflow_lite.ipynb +++ b/jupyter/tensorflow_lite/inference_with_tensorflow_lite.ipynb @@ -22,13 +22,13 @@ "source": [ "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n", "\n", - "%maven ai.djl:api:0.16.0\n", - "%maven ai.djl:model-zoo:0.16.0\n", - "%maven ai.djl.tflite:tflite-engine:0.16.0\n", + "%maven ai.djl:api:0.17.0\n", + "%maven ai.djl:model-zoo:0.17.0\n", + "%maven ai.djl.tflite:tflite-engine:0.17.0\n", "%maven org.slf4j:slf4j-simple:1.7.32\n", "\n", "// Use secondary engine to help pre-processing and post-processing\n", - "%maven ai.djl.pytorch:pytorch-engine:0.16.0" + "%maven ai.djl.pytorch:pytorch-engine:0.17.0" ] }, { diff --git a/jupyter/transfer_learning_on_cifar10.ipynb b/jupyter/transfer_learning_on_cifar10.ipynb index dfbc409a932..d65cc7a1786 100644 --- a/jupyter/transfer_learning_on_cifar10.ipynb +++ b/jupyter/transfer_learning_on_cifar10.ipynb @@ -35,10 +35,10 @@ "source": [ "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n", "\n", - "%maven ai.djl:api:0.16.0\n", - "%maven ai.djl:basicdataset:0.16.0\n", - "%maven ai.djl.mxnet:mxnet-engine:0.16.0\n", - "%maven ai.djl.mxnet:mxnet-model-zoo:0.16.0\n", + "%maven ai.djl:api:0.17.0\n", + "%maven ai.djl:basicdataset:0.17.0\n", + "%maven ai.djl.mxnet:mxnet-engine:0.17.0\n", + "%maven ai.djl.mxnet:mxnet-model-zoo:0.17.0\n", "%maven org.slf4j:slf4j-simple:1.7.32" ] }, diff --git a/jupyter/tutorial/01_create_your_first_network.ipynb b/jupyter/tutorial/01_create_your_first_network.ipynb index 35949f26de0..55b62459d59 100644 --- a/jupyter/tutorial/01_create_your_first_network.ipynb +++ b/jupyter/tutorial/01_create_your_first_network.ipynb @@ -27,7 +27,7 @@ "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n", "\n", "// Add the maven dependencies\n", - "%maven ai.djl:api:0.16.0\n", + "%maven ai.djl:api:0.17.0\n", "%maven org.slf4j:slf4j-simple:1.7.32" ] }, @@ -53,7 +53,7 @@ "\n", "## Application\n", "\n", - "The first thing to figure out when trying to build a neural network, like building most functions, is what your function signature is. What are your input types and output types? Because most models use relatively consistent signatures, we refer to them as [Applications](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/Application.html). Within the Applications interface, you can find a list of some of the more common model applications used in deep learning.\n", + "The first thing to figure out when trying to build a neural network, like building most functions, is what your function signature is. What are your input types and output types? Because most models use relatively consistent signatures, we refer to them as [Applications](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/Application.html). Within the Applications interface, you can find a list of some of the more common model applications used in deep learning.\n", "\n", "In this tutorial, we will focus on the image classification application. It is one of the most common first applications and has a significant history with deep learning. In image classification, the input is a single image and it is classified based on the main subject of the image into a number of different possible classes. The classes for the image depend on the specific data you are training with." ] @@ -83,7 +83,7 @@ "\n", "![Mnist Image](https://upload.wikimedia.org/wikipedia/commons/2/27/MnistExamples.png)\n", "\n", - "Once you understand your dataset, you should create an implementation of the [Dataset class](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/training/dataset/Dataset.html). In this case, we provide the MNIST dataset built-in to make it easy for you to use it.\n", + "Once you understand your dataset, you should create an implementation of the [Dataset class](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/training/dataset/Dataset.html). In this case, we provide the MNIST dataset built-in to make it easy for you to use it.\n", "\n", "## Multilayer Perceptron\n", "\n", @@ -123,16 +123,16 @@ "\n", "### NDArray\n", "\n", - "The core data type used for working with deep learning is the [NDArray](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/ndarray/NDArray.html). An NDArray represents a multidimensional, fixed-size homogeneous array. It has very similar behavior to the Numpy python package with the addition of efficient computing. We also have a helper class, the [NDList](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/ndarray/NDList.html) which is a list of NDArrays which can have different sizes and data types.\n", + "The core data type used for working with deep learning is the [NDArray](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/ndarray/NDArray.html). An NDArray represents a multidimensional, fixed-size homogeneous array. It has very similar behavior to the Numpy python package with the addition of efficient computing. We also have a helper class, the [NDList](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/ndarray/NDList.html) which is a list of NDArrays which can have different sizes and data types.\n", "\n", "### Block API\n", "\n", - "In DJL, [Blocks](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/nn/Block.html) serve a purpose similar to functions that convert an input `NDList` to an output `NDList`. They can represent single operations, parts of a neural network, and even the whole neural network. What makes blocks special is that they contain a number of parameters that are used in their function and are trained during deep learning. As these parameters are trained, the function represented by the blocks get more and more accurate.\n", + "In DJL, [Blocks](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/nn/Block.html) serve a purpose similar to functions that convert an input `NDList` to an output `NDList`. They can represent single operations, parts of a neural network, and even the whole neural network. What makes blocks special is that they contain a number of parameters that are used in their function and are trained during deep learning. As these parameters are trained, the function represented by the blocks get more and more accurate.\n", "\n", "When building these block functions, the easiest way is to use composition. Similar to how functions are built by calling other functions, blocks can be built by combining other blocks. We refer to the containing block as the parent and the sub-blocks as the children.\n", "\n", "\n", - "We provide several helpers to make it easy to build common block composition structures. For the MLP we will use the [SequentialBlock](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/nn/SequentialBlock.html), a container block whose children form a chain of blocks where each child block feeds its output to the next child block in a sequence.\n", + "We provide several helpers to make it easy to build common block composition structures. For the MLP we will use the [SequentialBlock](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/nn/SequentialBlock.html), a container block whose children form a chain of blocks where each child block feeds its output to the next child block in a sequence.\n", " " ] }, @@ -151,7 +151,7 @@ "source": [ "## Step 4: Add blocks to SequentialBlock\n", "\n", - "An MLP is organized into several layers. Each layer is composed of a [Linear Block](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/nn/core/Linear.html) and a non-linear activation function. If we just had two linear blocks in a row, it would be the same as a combined linear block ($f(x) = W_2(W_1x) = (W_2W_1)x = W_{combined}x$). An activation is used to intersperse between the linear blocks to allow them to represent non-linear functions. We will use the popular [ReLU](https://javadoc.io/static/ai.djl/api/0.16.0/ai/djl/nn/Activation.html#reluBlock--) as our activation function.\n", + "An MLP is organized into several layers. Each layer is composed of a [Linear Block](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/nn/core/Linear.html) and a non-linear activation function. If we just had two linear blocks in a row, it would be the same as a combined linear block ($f(x) = W_2(W_1x) = (W_2W_1)x = W_{combined}x$). An activation is used to intersperse between the linear blocks to allow them to represent non-linear functions. We will use the popular [ReLU](https://javadoc.io/static/ai.djl/api/0.17.0/ai/djl/nn/Activation.html#reluBlock--) as our activation function.\n", "\n", "The first layer and last layers have fixed sizes depending on your desired input and output size. However, you are free to choose the number and sizes of the middle layers in the network. We will create a smaller MLP with two middle layers that gradually decrease the size. Typically, you would experiment with different values to see what works the best on your data set." ] diff --git a/jupyter/tutorial/02_train_your_first_model.ipynb b/jupyter/tutorial/02_train_your_first_model.ipynb index e4e82509eed..e7d5972d8fd 100644 --- a/jupyter/tutorial/02_train_your_first_model.ipynb +++ b/jupyter/tutorial/02_train_your_first_model.ipynb @@ -23,10 +23,10 @@ "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n", "\n", "// Add the maven dependencies\n", - "%maven ai.djl:api:0.16.0\n", - "%maven ai.djl:basicdataset:0.16.0\n", - "%maven ai.djl:model-zoo:0.16.0\n", - "%maven ai.djl.mxnet:mxnet-engine:0.16.0\n", + "%maven ai.djl:api:0.17.0\n", + "%maven ai.djl:basicdataset:0.17.0\n", + "%maven ai.djl:model-zoo:0.17.0\n", + "%maven ai.djl.mxnet:mxnet-engine:0.17.0\n", "%maven org.slf4j:slf4j-simple:1.7.32" ] }, @@ -59,13 +59,13 @@ "source": [ "# Step 1: Prepare MNIST dataset for training\n", "\n", - "In order to train, you must create a [Dataset class](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/training/dataset/Dataset.html) to contain your training data. A dataset is a collection of sample input/output pairs for the function represented by your neural network. Each single input/output is represented by a [Record](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/training/dataset/Record.html). Each record could have multiple arrays of inputs or outputs such as an image question and answer dataset where the input is both an image and a question about the image while the output is the answer to the question.\n", + "In order to train, you must create a [Dataset class](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/training/dataset/Dataset.html) to contain your training data. A dataset is a collection of sample input/output pairs for the function represented by your neural network. Each single input/output is represented by a [Record](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/training/dataset/Record.html). Each record could have multiple arrays of inputs or outputs such as an image question and answer dataset where the input is both an image and a question about the image while the output is the answer to the question.\n", "\n", - "Because data learning is highly parallelizable, training is often done not with a single record at a time, but a [Batch](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/training/dataset/Batch.html). This can lead to significant performance gains, especially when working with images\n", + "Because data learning is highly parallelizable, training is often done not with a single record at a time, but a [Batch](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/training/dataset/Batch.html). This can lead to significant performance gains, especially when working with images\n", "\n", "## Sampler\n", "\n", - "Then, we must decide the parameters for loading data from the dataset. The only parameter we need for MNIST is the choice of [Sampler](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/training/dataset/Sampler.html). The sampler decides which and how many element from datasets are part of each batch when iterating through it. We will have it randomly shuffle the elements for the batch and use a batchSize of 32. The batchSize is usually the largest power of 2 that fits within memory." + "Then, we must decide the parameters for loading data from the dataset. The only parameter we need for MNIST is the choice of [Sampler](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/training/dataset/Sampler.html). The sampler decides which and how many element from datasets are part of each batch when iterating through it. We will have it randomly shuffle the elements for the batch and use a batchSize of 32. The batchSize is usually the largest power of 2 that fits within memory." ] }, { @@ -85,7 +85,7 @@ "source": [ "# Step 2: Create your Model\n", "\n", - "Next we will build a model. A [Model](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/Model.html) contains a neural network [Block](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/nn/Block.html) along with additional artifacts used for the training process. It possesses additional information about the inputs, outputs, shapes, and data types you will use. Generally, you will use the Model once you have fully completed your Block.\n", + "Next we will build a model. A [Model](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/Model.html) contains a neural network [Block](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/nn/Block.html) along with additional artifacts used for the training process. It possesses additional information about the inputs, outputs, shapes, and data types you will use. Generally, you will use the Model once you have fully completed your Block.\n", "\n", "In this part of the tutorial, we will use the built-in Multilayer Perceptron Block from the Model Zoo. To learn how to build it from scratch, see the previous tutorial: [Create Your First Network](01_create_your_first_network.ipynb).\n", "\n", @@ -108,21 +108,21 @@ "source": [ "# Step 3: Create a Trainer\n", "\n", - "Now, you can create a [`Trainer`](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/training/Trainer.html) to train your model. The trainer is the main class to orchestrate the training process. Usually, they will be opened using a try-with-resources and closed after training is over.\n", + "Now, you can create a [`Trainer`](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/training/Trainer.html) to train your model. The trainer is the main class to orchestrate the training process. Usually, they will be opened using a try-with-resources and closed after training is over.\n", "\n", "The trainer takes an existing model and attempts to optimize the parameters inside the model's Block to best match the dataset. Most optimization is based upon [Stochastic Gradient Descent](https://en.wikipedia.org/wiki/Stochastic_gradient_descent) (SGD).\n", "\n", "## Step 3.1: Setup your training configurations\n", "\n", - "Before you create your trainer, we we will need a [training configuration](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/training/DefaultTrainingConfig.html) that describes how to train your model.\n", + "Before you create your trainer, we we will need a [training configuration](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/training/DefaultTrainingConfig.html) that describes how to train your model.\n", "\n", "The following are a few common items you may need to configure your training:\n", "\n", - "* **REQUIRED** [`Loss`](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/training/loss/Loss.html) function: A loss function is used to measure how well our model matches the dataset. Because the lower value of the function is better, it's called the \"loss\" function. The Loss is the only required argument to the model\n", - "* [`Evaluator`](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/training/evaluator/Evaluator.html) function: An evaluator function is also used to measure how well our model matches the dataset. Unlike the loss, they are only there for people to look at and are not used for optimizing the model. Since many losses are not as intuitive, adding other evaluators such as Accuracy can help to understand how your model is doing. If you know of any useful evaluators, we recommend adding them.\n", - "* [`Training Listeners`](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/repository/zoo/ZooModel.html): The training listener adds additional functionality to the training process through a listener interface. This can include showing training progress, stopping early if training becomes undefined, or recording performance metrics. We offer several easy sets of [default listeners](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/repository/zoo/ZooModel.html).\n", + "* **REQUIRED** [`Loss`](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/training/loss/Loss.html) function: A loss function is used to measure how well our model matches the dataset. Because the lower value of the function is better, it's called the \"loss\" function. The Loss is the only required argument to the model\n", + "* [`Evaluator`](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/training/evaluator/Evaluator.html) function: An evaluator function is also used to measure how well our model matches the dataset. Unlike the loss, they are only there for people to look at and are not used for optimizing the model. Since many losses are not as intuitive, adding other evaluators such as Accuracy can help to understand how your model is doing. If you know of any useful evaluators, we recommend adding them.\n", + "* [`Training Listeners`](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/repository/zoo/ZooModel.html): The training listener adds additional functionality to the training process through a listener interface. This can include showing training progress, stopping early if training becomes undefined, or recording performance metrics. We offer several easy sets of [default listeners](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/repository/zoo/ZooModel.html).\n", "\n", - "You can also configure other options such as the Device, Initializer, and Optimizer. See [more details](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/training/TrainingConfig.html)." + "You can also configure other options such as the Device, Initializer, and Optimizer. See [more details](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/training/TrainingConfig.html)." ] }, { diff --git a/jupyter/tutorial/03_image_classification_with_your_model.ipynb b/jupyter/tutorial/03_image_classification_with_your_model.ipynb index 36c11888ba4..587aeea87a7 100644 --- a/jupyter/tutorial/03_image_classification_with_your_model.ipynb +++ b/jupyter/tutorial/03_image_classification_with_your_model.ipynb @@ -26,10 +26,10 @@ "// %mavenRepo snapshots https://oss.sonatype.org/content/repositories/snapshots/\n", "\n", "// Add the maven dependencies\n", - "%maven ai.djl:api:0.16.0\n", - "%maven ai.djl:model-zoo:0.16.0\n", - "%maven ai.djl.mxnet:mxnet-engine:0.16.0\n", - "%maven ai.djl.mxnet:mxnet-model-zoo:0.16.0\n", + "%maven ai.djl:api:0.17.0\n", + "%maven ai.djl:model-zoo:0.17.0\n", + "%maven ai.djl.mxnet:mxnet-engine:0.17.0\n", + "%maven ai.djl.mxnet:mxnet-model-zoo:0.17.0\n", "%maven org.slf4j:slf4j-simple:1.7.32" ] }, @@ -100,7 +100,7 @@ "\n", "## Step 3: Create a `Translator`\n", "\n", - "The [`Translator`](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/translate/Translator.html) is used to encapsulate the pre-processing and post-processing functionality of your application. The input to the processInput and processOutput should be single data items, not batches." + "The [`Translator`](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/translate/Translator.html) is used to encapsulate the pre-processing and post-processing functionality of your application. The input to the processInput and processOutput should be single data items, not batches." ] }, { @@ -141,7 +141,7 @@ "source": [ "## Step 4: Create Predictor\n", "\n", - "Using the translator, we will create a new [`Predictor`](https://javadoc.io/static/ai.djl/api/0.16.0/index.html?ai/djl/inference/Predictor.html). The predictor is the main class to orchestrate the inference process. During inference, a trained model is used to predict values, often for production use cases. The predictor is NOT thread-safe, so if you want to do prediction in parallel, you should call newPredictor multiple times to create a predictor object for each thread." + "Using the translator, we will create a new [`Predictor`](https://javadoc.io/static/ai.djl/api/0.17.0/index.html?ai/djl/inference/Predictor.html). The predictor is the main class to orchestrate the inference process. During inference, a trained model is used to predict values, often for production use cases. The predictor is NOT thread-safe, so if you want to do prediction in parallel, you should call newPredictor multiple times to create a predictor object for each thread." ] }, { @@ -159,7 +159,7 @@ "source": [ "## Step 5: Run inference\n", "\n", - "With our predictor, we can simply call the [predict](https://javadoc.io/static/ai.djl/api/0.16.0/ai/djl/inference/Predictor.html#predict-I-) method to run inference. For better performance, you can also call [batchPredict](https://javadoc.io/static/ai.djl/api/0.16.0/ai/djl/inference/Predictor.html#batchPredict-java.util.List-) with a list of input items. Afterwards, the same predictor should be used for further inference calls. " + "With our predictor, we can simply call the [predict](https://javadoc.io/static/ai.djl/api/0.17.0/ai/djl/inference/Predictor.html#predict-I-) method to run inference. For better performance, you can also call [batchPredict](https://javadoc.io/static/ai.djl/api/0.17.0/ai/djl/inference/Predictor.html#batchPredict-java.util.List-) with a list of input items. Afterwards, the same predictor should be used for further inference calls. " ] }, { diff --git a/model-zoo/README.md b/model-zoo/README.md index bee9f15f799..111ff70f032 100644 --- a/model-zoo/README.md +++ b/model-zoo/README.md @@ -33,7 +33,7 @@ You can pull the model zoo from the central Maven repository by including the fo ai.djl model-zoo - 0.16.0 + 0.17.0 ```