diff --git a/content/modules/ROOT/assets/images/05/01-openshift-console.png b/content/modules/ROOT/assets/images/05/01-openshift-console.png index 636e31d9..f37bd408 100644 Binary files a/content/modules/ROOT/assets/images/05/01-openshift-console.png and b/content/modules/ROOT/assets/images/05/01-openshift-console.png differ diff --git a/content/modules/ROOT/pages/01-01-setting-stage.adoc b/content/modules/ROOT/pages/01-01-setting-stage.adoc index 4f866d5c..b39d2b92 100644 --- a/content/modules/ROOT/pages/01-01-setting-stage.adoc +++ b/content/modules/ROOT/pages/01-01-setting-stage.adoc @@ -14,6 +14,6 @@ In order to make this lab seem more realistic, we will be describing this imagin ** provide advice on potential improvements * That team is currently very small (about 4 people) ** Findings will be presented to the board -** If they are convincing, we'll be granted the resources to implement the recommendations -* The next slides are the materials that were presented to the board +** If they are convincing, the team will be granted the resources to implement the recommendations +* The next sections in this chapter are the materials that were presented to the board diff --git a/content/modules/ROOT/pages/01-03-proposed-improvements.adoc b/content/modules/ROOT/pages/01-03-proposed-improvements.adoc index 2758ae5a..32f4cc33 100644 --- a/content/modules/ROOT/pages/01-03-proposed-improvements.adoc +++ b/content/modules/ROOT/pages/01-03-proposed-improvements.adoc @@ -16,5 +16,5 @@ include::_attributes.adoc[] ** improve fraud detection by 25% * Requirements ** More precise measurements of performance -** at baseline -** after every change/improvement +**** at baseline +**** after every change/improvement diff --git a/content/modules/ROOT/pages/01-04-examples-from-prototype.adoc b/content/modules/ROOT/pages/01-04-examples-from-prototype.adoc index 8766521c..2a2f9d61 100644 --- a/content/modules/ROOT/pages/01-04-examples-from-prototype.adoc +++ b/content/modules/ROOT/pages/01-04-examples-from-prototype.adoc @@ -2,7 +2,6 @@ include::_attributes.adoc[] The examples below are what we hope to achieve through our prototype version of the improved process. -Some are mockups, and some are screenshots from the prototype application. == Using an LLM for text summarization diff --git a/content/modules/ROOT/pages/01-05-results.adoc b/content/modules/ROOT/pages/01-05-results.adoc index f3dca4d2..11cf62cc 100644 --- a/content/modules/ROOT/pages/01-05-results.adoc +++ b/content/modules/ROOT/pages/01-05-results.adoc @@ -10,7 +10,8 @@ include::_attributes.adoc[] == Next steps -We know that you all have various background and responsibilities. (Coders, IT, DevOps, Data Scientists, etc..) +We know that you all have various background and responsibilities (Coders, IT, DevOps, Data Scientists, etc..). + But before you start working in your narrow area of specialization, we want you all to have a decent understanding of all the work that was done during the prototype phase. -The next 75 minutes is your official training session, for you to ramp up and gain an understanding for the various technologies involved during the prototype phase. +The next 75 minutes is your official training session, for you to ramp up and gain an understanding of the various technologies involved during the prototype phase. diff --git a/content/modules/ROOT/pages/02-01-getting-connected.adoc b/content/modules/ROOT/pages/02-01-getting-connected.adoc index da2a2ad1..e3a772dc 100644 --- a/content/modules/ROOT/pages/02-01-getting-connected.adoc +++ b/content/modules/ROOT/pages/02-01-getting-connected.adoc @@ -5,18 +5,6 @@ For the purposes of this training session, we have provisioned a single OpenShif Each person attending this lab will have a unique user account in which to do their work. -== Getting Support during RH1 In-Person Labs - -* In the room -** Some very kind colleagues will be walking around in the room, to help and/or answer questions. -** If you run into a particular issue, call out to one of them and quietly explain what the issue is. -** If they are unsure or if it's likely to be a long explanation, they might ask you to "post the question in slack" instead. (see below) -* Over slack -** We have a dedicated Slack Channel where more colleagues (who kindly agreed to stay up late) will try to answer questions. -** Head over to the slack channel called https://redhat.enterprise.slack.com/archives/C066EQ8LWBS[#rh1-insurance-claims-lab,window=_blank] -** Post a message such as `I have an issue with exercise number 2.4` -** Use the https://slack.com/help/articles/115000769927-Use-threads-to-organize-discussions-[threads,window=_blank] feature in slack to then post a screenshot of a description of the issue, in-thread. - == Environment information If you are using the customized version of the instructions, the information below will render properly. If not, you will see placeholder values instead. @@ -26,27 +14,15 @@ If you are using the customized version of the instructions, the information bel In a new window or tab, open the following URL and log in: -// * OpenShift Console URL: {openshift_console_url}[{openshift_console_url},window=_blank] - -// * and after that: -// + -// image::02/02-01-login2.png[] - * The Red Hat OpenShift AI Dashboard URL for our shared environment: ** https://rhods-dashboard-redhat-ods-applications.{openshift_cluster_ingress_domain}/[https://rhods-dashboard-redhat-ods-applications.{openshift_cluster_ingress_domain}/,window=_blank] - -// * Alternatively, you can access Red Hat OpenShift AI from the OpenShift Console by clicking on: -// + -// [.bordershadow] -// image::02/02-01-access-rhoai.png[] * Enter your credentials (as detailed above) * The result should look like: + [.bordershadow] -image::02/02-01-login1.png[] - -* Because the password is so simple (`{password}`), Chrome might display a scary message such as: +image::02/02-01-login1.png[width=50%] +* Because the password is so simple (`{password}`), your browser might display a scary message such as: + [.bordershadow] image::02/02-01-login-scary.png[] @@ -57,19 +33,18 @@ image::02/02-01-login-scary.png[] [.bordershadow] image::02/02-01-rhoai-front-page.png[] -If you got this far and saw all that, congratulations, you got properly connected to the OpenShift AI Dashboard Application! - -// * The `oc login` command: -// [.lines_space] -// [.console-input] -// [source, text] -// [subs=attributes+] -// {login_command} +If you got this far and saw all that, congratulations, you properly connected to the OpenShift AI Dashboard Application! -// == Other credentials and URLs: +We are now ready to start the lab. -// TODO? +== Getting Support during RH1 In-Person Labs -// * ArgoCD view -// * Gitea -// * Database +* In the room +** Some very kind colleagues will be walking around in the room, to help and/or answer questions. +** If you run into a particular issue, call out to one of them and quietly explain what the issue is. +** If they are unsure or if it's likely to be a long explanation, they might ask you to "post the question in slack" instead. (see below) +* Over slack +** We have a dedicated Slack Channel where more colleagues (who kindly agreed to stay up late) will try to answer questions. +** Head over to the slack channel called https://redhat.enterprise.slack.com/archives/C066EQ8LWBS[#rh1-insurance-claims-lab,window=_blank] +** Post a message such as `I am userX and I have an issue with exercise number 2.4` +** Use the https://slack.com/help/articles/115000769927-Use-threads-to-organize-discussions-[threads,window=_blank] feature in slack to then post a screenshot of a description of the issue, in-thread. diff --git a/content/modules/ROOT/pages/02-02-creating-project.adoc b/content/modules/ROOT/pages/02-02-creating-project.adoc index 9c1078dc..a6aba725 100644 --- a/content/modules/ROOT/pages/02-02-creating-project.adoc +++ b/content/modules/ROOT/pages/02-02-creating-project.adoc @@ -7,7 +7,7 @@ As a preliminary step, each of you is going to ** this will help keep your things together . Create a Data Connection -** we need that for the pipeline server to store it artifacts +** we need that for the pipeline server to store its artifacts . Deploy a Data Science Pipeline Server ** we will need one, and it's better to create it from the start @@ -16,7 +16,7 @@ As a preliminary step, each of you is going to ** we will use it to review content and notebooks . Clone the git repo into your Workbench -** this contains all the code we've been using in our prototyping efforts +** this contains all the code from the prototype The instructions below will guide you through these steps. Follow them carefully. @@ -27,31 +27,30 @@ The instructions below will guide you through these steps. Follow them carefully [.bordershadow] image::02/02-02-ds-proj-nav.png[] -* Create a project with the **same name** as your user id. +* Create a project with the **same name** as your user id ** You have been assigned a unique user ID: `{user}` ** You need to now create a project with the exact same name: `{user}` - ++ IMPORTANT: Your assigned user is {user}. Don't mess that up or things will break later on * Leave the resource name unchanged -* Enter your first and last name in the description of the project. - +* Optionally, enter your first and last name in the description of the project. * It should look like this: + [.bordershadow] image::02/02-02-create-project.png[] - ++ IMPORTANT: It should **NOT** be `userX` like in the screenshot. (for you, `X` should be a number instead) == Create a Data Connection for the pipeline server * We have deployed an instance of Minio in the cluster to act as a simple Object Storage for our purposes. -* We will need to **Add data connection** that points to it. +* You will need to **Add data connection** that points to it. + [.bordershadow] image::02/02-02-add-dc.png[] -* Here is the info you need to enter: +* Here is the information you need to enter: ** Name: [.lines_space] [.console-input] @@ -88,7 +87,7 @@ none [source, text] [subs=attributes+] {user} - ++ IMPORTANT: Once again, the bucket you will use has to match with the user ID you were provided * The result should look like: @@ -100,12 +99,12 @@ image::02/data-connection.png[] It is highly recommended to create your pipeline server before creating a workbench. So let's do that now! -* In your DS project, click on **Create a pipeline Server** +* In your Data Science Project (DSP), click on **Create a pipeline Server** + [.bordershadow] image::02/02-02-pipelineserver01.png[] -* Select the Data Connection created earlier (**Shared Minio - pipelines**) and click the **configure** button: +* Select the Data Connection created earlier (**Shared Minio - pipelines**) and click the **Configure** button: + [.bordershadow] image::02/02-02-pipelineserver02.png[] diff --git a/content/modules/ROOT/pages/02-03-creating-workbench.adoc b/content/modules/ROOT/pages/02-03-creating-workbench.adoc index 9b2662cd..fb4554f9 100644 --- a/content/modules/ROOT/pages/02-03-creating-workbench.adoc +++ b/content/modules/ROOT/pages/02-03-creating-workbench.adoc @@ -11,8 +11,8 @@ image::02/02-03-create-wb.png[] * Make sure it has the following characteristics: ** Choose a name for it, like: `My Workbench` ** Image Selection: `CUSTOM - Insurance Claim Processing Lab Workbench` -** Container Size: Standard -** Accelerator: None +** Container Size: `Standard` +** Accelerator: `None` * That should look like: + [.bordershadow] @@ -30,7 +30,7 @@ image::02/02-03-open-link.png[] [.bordershadow] image::02/02-02-accept.png[] -* Do so. +* Do so * You should now see this: + [.bordershadow] @@ -41,7 +41,7 @@ image::02/02-02-jupyter.png[] We will clone the content of our Git repo so that you can access all the materials that were created as part of our prototyping exercise. * Using the Git UI: -** If you want to use the git UI in Jupyter: +** Open the Git UI in Jupyter: + [.bordershadow] image::02/git-clone-1.png[] @@ -58,21 +58,4 @@ image::02/git-clone-1.png[] [.bordershadow] image::02/git-clone-2.png[] -// -- -// Using the Git CLI:: -// + -// -- -// If you prefer using the `git` CLI, open a a terminal in JupyterLab, and paste this code: - -// [.console-input] -// [source,adoc] -// [subs=attributes+] -// ---- -// git clone {git-clone-repo-url} -// cd insurance-claim-processing/ -// git checkout {git-clone-repo-branch} -// ---- -// -- -// ==== - At this point, your project is ready for the work we want to do in it. diff --git a/content/modules/ROOT/pages/03-01-notebook-based-llm.adoc b/content/modules/ROOT/pages/03-01-notebook-based-llm.adoc index 96d8277c..031e84e6 100644 --- a/content/modules/ROOT/pages/03-01-notebook-based-llm.adoc +++ b/content/modules/ROOT/pages/03-01-notebook-based-llm.adoc @@ -3,6 +3,6 @@ include::_attributes.adoc[] In this exercise, we will use a notebook to investigate how LLMs can be used. -From the `lab-materials/03` folder, please open the notebook called `03-01-nb-llm-example.ipynb` and follow the instructions. +From the `insurance-claim-processing/lab-materials/03` folder, please open the notebook called `03-01-nb-llm-example.ipynb` and follow the instructions. -When done, you can close the notebook and head for the next page. +When done, you can close the notebook and head to the next page. diff --git a/content/modules/ROOT/pages/03-02-summarization.adoc b/content/modules/ROOT/pages/03-02-summarization.adoc index ad0ac1ec..6fe5c35b 100644 --- a/content/modules/ROOT/pages/03-02-summarization.adoc +++ b/content/modules/ROOT/pages/03-02-summarization.adoc @@ -3,6 +3,6 @@ include::_attributes.adoc[] As part of our claim processing, we want to summarize the text of a claim. In the next notebook we will investigate how the LLM can be used to do this task. -From the `lab-materials/03` folder, please open the notebook called `03-02-summarization.ipynb` and follow the instructions. +From the `insurance-claim-processing/lab-materials/03` folder, please open the notebook called `03-02-summarization.ipynb` and follow the instructions. -When done, you can close the notebook and head for the next page. +When done, you can close the notebook and head to the next page. diff --git a/content/modules/ROOT/pages/03-03-information-extractions.adoc b/content/modules/ROOT/pages/03-03-information-extractions.adoc index a6feb7cd..60e65537 100644 --- a/content/modules/ROOT/pages/03-03-information-extractions.adoc +++ b/content/modules/ROOT/pages/03-03-information-extractions.adoc @@ -3,6 +3,6 @@ include::_attributes.adoc[] As part of our claim processing, we want to extract some information from the text of a claim. In the next notebook we will investigate how the LLM can be used to do this task. -From the `lab-materials/03` folder, please open the notebook called `03-03-information-extraction.ipynb` and follow the instructions. +From the `insurance-claim-processing/lab-materials/03` folder, please open the notebook called `03-03-information-extraction.ipynb` and follow the instructions. -When done, you can close the notebook and head for the next page. +When done, you can close the notebook and head to the next page. diff --git a/content/modules/ROOT/pages/03-04-comparing-models.adoc b/content/modules/ROOT/pages/03-04-comparing-models.adoc index 76369c26..24469054 100644 --- a/content/modules/ROOT/pages/03-04-comparing-models.adoc +++ b/content/modules/ROOT/pages/03-04-comparing-models.adoc @@ -3,9 +3,9 @@ include::_attributes.adoc[] So far, for this lab, we have used the model https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2[Mistral-7B Instruct v2,window=_blank]. Although lighter than other models, it is still quite heavy and we need a large GPU to run it. Would we get as good results with a smaller model? Let's try! -In this exercise, we'll pitch it against a much smaller LLM called https://huggingface.co/google/flan-t5-small)[flan-t5-small,window=_blank]. +In this exercise, we'll pitch it against a much smaller LLM called https://huggingface.co/google/flan-t5-small[flan-t5-small,window=_blank]. -From the `lab-materials/03` folder, please open the notebook called `03-04-comparing-models.ipynb` and follow the instructions. +From the `insurance-claim-processing/lab-materials/03` folder, please open the notebook called `03-04-comparing-models.ipynb` and follow the instructions. -When done, you can close the notebook and head for the next page. +When done, you can close the notebook and head to the next page. diff --git a/content/modules/ROOT/pages/03-05-prompt-engineering.adoc b/content/modules/ROOT/pages/03-05-prompt-engineering.adoc index da7a611f..e9a02141 100644 --- a/content/modules/ROOT/pages/03-05-prompt-engineering.adoc +++ b/content/modules/ROOT/pages/03-05-prompt-engineering.adoc @@ -3,28 +3,30 @@ include::_attributes.adoc[] NOTE: This part of the lab is marked as optional. It can therefore be skipped without effect on the other parts of the lab. You can always come back to it later at the end of the lab if you have time to spare. +IMPORTANT: When you interact with ChatGPT or other commercial services, a lot of guardrails are in place to prevent you from getting unwanted or not suitable for work results. In our Lab exercises, we are using a model that is not protected by those guardrails, and we will be modifying its settings. Therefore, it is your responsibility to do it in a safe way, and to make sure that the results you get are suitable for you and/or your audience. The authors of this Lab cannot be held responsible for any inappropriate results you may get. + As you have seen in the previous sections, there are different factors that will influence the results you can get from your model: the model parameters, the prompt, the query, and of course the data itself... Let's see how we can adjust those different factors to get the best results. == Modifying the settings -Go back to the notebook `03-01-nb-llm-example.ipynb` and try to change the following to see how it impacts the results: +Go back to the notebook `03-01-nb-llm-example.ipynb` and try changing: -- The llm parameters, like the `temperature`, to make the model more creative. +- The llm parameters, like the `temperature`, to make the model more creative - The `prompt template`, to make the model behave differently, like giving the answer in the form of a poem. Or more useful in an enterprise context, give answers like it was addressing different types of audience: for example 5 years old children, or people without technical knowledge, or the opposite... -- The `query` itself which you can use to override certain aspects of the prompt. This will show you why it's important to have guardrails in place against "prompt injection", like pre-filtering user queries. +- The `query` itself which you can use to override certain aspects of the prompt. This will show you why it's important to have guardrails in place against "prompt injection", like pre-filtering user queries == Modifying the data -Go back to the notebook `03-02-summarization.ipynb` and try to change the following to see how it impacts the results: +Go back to the notebook `03-02-summarization.ipynb` and try changing: - Edit or create new claims, making them harder to understand. (Note: to edit the json file in Jupyter, right click on it and select "Open With > Editor") -- Experiment with different languages. +- Experiment with different languages == Modifying the prompt -Go back to the notebook `03-03-information-extraction.ipynb` and try to change the following to see how it impacts the results: +Go back to the notebook `03-03-information-extraction.ipynb` and try changing: -- Edit or create new claims, making them harder to understand. -- Try to have the model more precise and concise in date and time extraction, like making it respect a specific format. +- Edit or create new claims, making them harder to understand +- Try to have the model more precise and concise in date and time extraction, like making it respect a specific format diff --git a/content/modules/ROOT/pages/03-06-sanity-check.adoc b/content/modules/ROOT/pages/03-06-sanity-check.adoc index 39daa44b..69a82427 100644 --- a/content/modules/ROOT/pages/03-06-sanity-check.adoc +++ b/content/modules/ROOT/pages/03-06-sanity-check.adoc @@ -2,22 +2,19 @@ include::_attributes.adoc[] == What will the pipeline do? -To make sure that everything works as we would expect it to, and that the LLM model has not been tampered with, we will create a sanity-check pipeline that tests the model through its endpoint. + +To make sure that everything works as we would expect it to, and that the model has not been tampered with, we will create a sanity-check pipeline that tests the model through its endpoint. + We will test the response time, the response quality, and that the model hash has not changed. + -And to make sure it remains the case, we'll schedule that pipeline. - -== Deploy a pipeline Server in your project - -This step was already done as part of the project setup. +And to make sure it stays the same over time, we'll schedule that pipeline. == Deploy a sanity-check pipeline -In the `lab-materials/03/06` folder there are two pipeline files, one *sanity_check.pipeline* and one *sanity_check.yaml* file. +In the `insurance-claim-processing/lab-materials/03/06` folder there are two pipeline files, one `sanity_check.pipeline` and one `sanity_check.yaml` file. The `.pipeline` file can be opened in Elyra to be visually modified and executed, while the `.yaml` file can be imported into the pipeline server through the RHOAI Dashboard. + Here we will be running the pipeline through Elyra. == Ad-Hoc execution + Running it through Elyra is the same as doing an ad-hoc execution of the pipeline, as opposed to importing the pipeline which won't automatically execute it. . Start by going to your running workbench @@ -28,7 +25,7 @@ Running it through Elyra is the same as doing an ad-hoc execution of the pipelin .. response time check .. security check . Feel free to peek into each of the python files by double clicking on the nodes to see what they do. + -. After the tests have been ran, we have a final function that will summarize the results and log them. +. After the tests have been run, we have a final function that will summarize the results and log them. . To run the pipeline, press the Play button in the menu bar. + [.bordershadow] @@ -42,7 +39,7 @@ image::03/03-07-run-pipeline-ok.png[] == Schedule execution -We can also **schedule** an execution so that the sanity check is ran at regular intervals. + +We can also **schedule** an execution so that the sanity check is executed at regular intervals. + To do that: . Go back to the OpenShift AI Data Science Project @@ -52,10 +49,10 @@ To do that: [.bordershadow] image::03/07-create-run.png[create run] -. On the next screen, choose a name, select a periodic run to trigger every **Day** and click **Create**: +. On the next screen, choose a name, select a Periodic run to trigger every **Day** and click **Create**: + [.bordershadow] image::03/03-06-dailyrun.png[] . We can now leave the sanity-check pipeline alone. -. It will run daily, and will inform us if anything goes wrong with our LLM model. \ No newline at end of file +. It will run daily, and will inform us if anything goes wrong with our LLM. \ No newline at end of file diff --git a/content/modules/ROOT/pages/04-01-over-approach.adoc b/content/modules/ROOT/pages/04-01-over-approach.adoc index dc02b89e..2199c76f 100644 --- a/content/modules/ROOT/pages/04-01-over-approach.adoc +++ b/content/modules/ROOT/pages/04-01-over-approach.adoc @@ -1,17 +1,17 @@ = Overall Approach include::_attributes.adoc[] -As part of this prototype, we investigated the use of the YOLOV8 model. +As part of this prototype, we investigated the use of the YOLOv8 model. This model can be found online at https://www.yolov8.com[yolov8,window=_blank] and downloaded. -We will first review its out-of-the-box capabilities. We will then fine-tune it to allow it to do more specialized work for us. Once we have a new, customized version of the model, we will deploy it in ModelMesh. Once that is done, we will send queries to it. +We will first review its out-of-the-box capabilities. We will then fine-tune it to allow it to do more specialized work for us. Once we have a new, customized version of the model, we will deploy it in OpenSift AI Model Serving. Once that is done, we will send queries to it. -Let's start by looking at a yolov8 model and explore how it works on static car images. +Let's start by looking at a YOLOv8 model and explore how it works on static car images. [.bordershadow] image::04/sample-car-image.png[car image] -- In your running workbench, navigate to the folder `lab-materials/04`. +- In your running workbench, navigate to the folder `insurance-claim-processing/lab-materials/04`. - Look for (and open) the notebook called `04-01-over-approach.ipynb` - Execute the cells of the notebook, and ensure you understand what is happening diff --git a/content/modules/ROOT/pages/04-02-car-recog.adoc b/content/modules/ROOT/pages/04-02-car-recog.adoc index 8c3f0011..6454cce9 100644 --- a/content/modules/ROOT/pages/04-02-car-recog.adoc +++ b/content/modules/ROOT/pages/04-02-car-recog.adoc @@ -6,7 +6,7 @@ In our last notebook we confirmed that the YOLO model could identify cars in a p [.bordershadow] image::04/sample-car-image.png[car image] -We also confirmed that while a YOLO model can identify multiple 'cars' in an image, humans may have a difficult time verifying the identified cars. +We discovered that YOLO is able to detect multiple cars in an image. However, we did not see which cars were identified. [.bordershadow] image::04/multiple-car-images.png[multiple cars] @@ -14,14 +14,14 @@ image::04/multiple-car-images.png[multiple cars] [.bordershadow] image::04/model-prediction-results.png[predict cars] -In the above image, the yolo model identified 17 cars and 1 truck. Which cars ,in the above image, were identified? +In the above image, the yolo model identified 17 cars and 1 truck. Therefore we need to write some code that will draw boxes around the cars identified by the YOLO model. [.bordershadow] image::04/box-identified-cars.png[identify cars] -- In your running pytorch workbench, navigate to the folder `lab-materials/04`. +- In your running workbench, navigate to the folder `insurance-claim-processing/lab-materials/04`. - Look for (and open) the notebook called `04-02-car-recognition.ipynb` - Execute the cells of the notebook, and ensure you understand what is happening diff --git a/content/modules/ROOT/pages/04-03-model-retraining.adoc b/content/modules/ROOT/pages/04-03-model-retraining.adoc index 91f81142..72e6031c 100644 --- a/content/modules/ROOT/pages/04-03-model-retraining.adoc +++ b/content/modules/ROOT/pages/04-03-model-retraining.adoc @@ -15,22 +15,11 @@ image::04/roboflow-test-images.png[roboflow images] 3. Each image has an annotation text file in the 'labels' subfolder. The annotation text files have the same names as the image files. ==== -We have provided the following 2 training data sets, available as 'zip files', and located in an S3 bucket: +We have provided the following training data set, available as a zip file, and located in an S3 bucket: `accident-sample.zip` (as we don,t have time in this Lab to fully retrain the model, we will use a sample of the training data set). -1. accident-full.zip - to be used to fully re-train the model. -2. accident-sample.zip - to be used to partially re-train the model when we don't have the time to fully re-train the model. - -Your instructor will let you know which data set 'zip file' you will be using in your lab. - -Once the images and associated annotations are ready, we create a dataset descriptor YAML file (data.yaml) that points to the created datasets and describes the object classes in them. This YAML file is passed to the 'train' method of the model to start the training process. - -[.bordershadow] -image::04/yaml-file.png[yaml file] - -- In your running workbench, navigate to the folder `lab-materials/04`. +- In your running workbench, navigate to the folder `insurance-claim-processing/lab-materials/04`. - Look for (and open) the notebook called `04-03-model-retraining.ipynb` -- Execute the cells of the notebook. But note that the actual training step would take way too long on CPU, so we will do the training on a sample only. Even with that, you can stop/terminate the kernel if you want if it takes too long to free up resources. - +- Execute the cells of the notebook. == Interpreting the Model re-Training Results [%collapsible] diff --git a/content/modules/ROOT/pages/04-04-accident-recog.adoc b/content/modules/ROOT/pages/04-04-accident-recog.adoc index 933b3d07..df2e0816 100644 --- a/content/modules/ROOT/pages/04-04-accident-recog.adoc +++ b/content/modules/ROOT/pages/04-04-accident-recog.adoc @@ -3,14 +3,15 @@ include::_attributes.adoc[] Now that we have retrained our model we can test it against some sample images. -We have converted our model to onnx format and placed a copy within an amazon s3 bucket. We will test this version against some sample test images. +We have converted our model to onnx format and placed a copy within an S3 bucket. We will test this version against some sample test images. - -Using the re-trained model, on the below image, we see that the model is able to identify a 'severe' car crash with '88%' probability. +Using the re-trained model, we will see that we are able to identify a severe car crash with 88% certainty, like in the below picture. [.bordershadow] image::04/retrained-model-results.png[retrained modelresults] -- In your running pytorch workbench, navigate to the folder `lab-materials/04`. +To test the model yourself: + +- In your workbench, navigate to the folder `insurance-claim-processing/lab-materials/04`. - Look for (and open) the notebook called `04-04-accident-recog.ipynb` - Execute the cells of the notebook. diff --git a/content/modules/ROOT/pages/04-05-model-serving.adoc b/content/modules/ROOT/pages/04-05-model-serving.adoc index 67536de1..e7075386 100644 --- a/content/modules/ROOT/pages/04-05-model-serving.adoc +++ b/content/modules/ROOT/pages/04-05-model-serving.adoc @@ -4,7 +4,7 @@ include::_attributes.adoc[] . At this point, we need to deploy the model into RHOAI model serving. . We will create another data connection... .. with almost identical information -.. but we will change the bucket name from `userX` to 'models' +.. but we will change the bucket name from `userX` to `models` == Create a Data Connection @@ -56,7 +56,7 @@ image::04/model-data-connection.png[model connection] In your project create a model server. -* Click **Add server** +* Click **Add model server** + [.bordershadow] image::04/add-model-server.png[] @@ -116,7 +116,7 @@ image::04/add-model-server-config.png[] == Deploy the Model -In your project, under 'Models and model servers' select 'Deploy model'. +In your project, under **Models and model servers** select **Deploy model**. * Click **Deploy model** + @@ -169,7 +169,7 @@ image::04/model-deployed-success.png[] We will now confirm that the model is indeed working by querying it! -== Using the served Model +== Querying the served Model Once the model is served, we can use it as an endpoint that can be queried. We'll send a request to it, and get a result. And unlike our earlier notebook-based version, this applies to anyone working within our cluster. This could either be colleagues, or applications. @@ -180,11 +180,11 @@ Once the model is served, we can use it as an endpoint that can be queried. We'l [.bordershadow] image::04/inference-url.png[] -* Note or copy the **RestUrl**, which should be something like `http://modelmesh-serving.{user}:8008` +* Note or copy the **RestUrl**, which should be something like `\http://modelmesh-serving.{user}:8008` We will now use this URL to query the model. -- In your running workbench, navigate to the folder `lab-materials/04`. +- In your running workbench, navigate to the folder `insurance-claim-processing/lab-materials/04`. - Look for (and open) the notebook called `04-05-model-serving.ipynb`. - Execute the cells of the notebook, and ensure you understand what is happening. diff --git a/content/modules/ROOT/pages/05-01-application.adoc b/content/modules/ROOT/pages/05-01-application.adoc index fc77ece5..baea214a 100644 --- a/content/modules/ROOT/pages/05-01-application.adoc +++ b/content/modules/ROOT/pages/05-01-application.adoc @@ -23,9 +23,9 @@ If you want to have a look at the code of the application, you can do it directl - Close the different Jupyter tabs that may still be opened in your browser. - Go back to the OpenShift AI dashboard. -- Stop your Workbench by using the toggle button. +- Stop your Workbench by using the toggle. -- You should probably give it 5 or 10 seconds to fully stop. -- Edit the Workbench configuration by clicking on the menu item at the left, then **Edit** button. +- Edit the Workbench configuration by clicking on the menu item at the left, then **Edit workbench** button. + [.bordershadow] image::05/stop-workbench.png[] @@ -41,7 +41,7 @@ image::05/vscode-image.png[] [.bordershadow] image::05/update-workbench.png[] -- You can now start your Workbench again by using the toggle button. +- You can now start your Workbench again by using the toggle. - Once the Workbench is started, click on the **Open** Link. - You will be redirected to the VSCode IDE. - After logging in and accepting the authorization, VSCode will open. @@ -51,11 +51,11 @@ image::05/update-workbench.png[] [.bordershadow] image::05/trust-vscode.png[] -- You can also close the Welcome tab, and you are in a familiar VSCode environment. -- Your Persistent Volume has been automatically reconnected to this new environment. --- Therefore, all the data you created in the Jupyter environment is still available --- You can see more files because VSCode displays hidden files by default, which Jupyter does not do. -- If you want, you can now explore the code of the application, in the `app` folder: +* You can also close the Welcome tab, and you are in a familiar VSCode environment. +* Your Persistent Volume has been automatically reconnected to this new environment. +** Therefore, all the data you created in the Jupyter environment is still available +** You can see more files because VSCode displays hidden files by default, which Jupyter does not do. +* If you want, you can now explore the code of the application, in the `app` folder: + [.bordershadow] image::05/app-code.png[] diff --git a/content/modules/ROOT/pages/05-02-openshift-terminal.adoc b/content/modules/ROOT/pages/05-02-openshift-terminal.adoc index b60bd272..72547e04 100644 --- a/content/modules/ROOT/pages/05-02-openshift-terminal.adoc +++ b/content/modules/ROOT/pages/05-02-openshift-terminal.adoc @@ -1,8 +1,8 @@ = Working with the OpenShift Terminal include::_attributes.adoc[] -To deploy the application, you will use the OpenShift Terminal. + -This is a web-based terminal that allows you to execute commands on the OpenShift cluster. +To deploy the application, you will use the OpenShift Web Terminal. + +This is a web-based terminal that allows you to execute commands on the OpenShift cluster from the OpenShift Console. == Opening the OpenShift Console diff --git a/content/modules/ROOT/pages/05-03-web-app-deploy-application.adoc b/content/modules/ROOT/pages/05-03-web-app-deploy-application.adoc index f11210c1..8a68e210 100644 --- a/content/modules/ROOT/pages/05-03-web-app-deploy-application.adoc +++ b/content/modules/ROOT/pages/05-03-web-app-deploy-application.adoc @@ -4,9 +4,9 @@ include::_attributes.adoc[] == Deploy your instance of ArgoCD We will start by deploying an instance of ArgoCD in your namespace. + -This will be used to deploy the rest of the applications. +This will be used to deploy the application. -- Copy the content of the following text, and paste it in the OpenShift Terminal to deploy ArgoCD. +- Copy the following text, and paste it in the OpenShift Terminal to deploy ArgoCD. + [.lines_space] [.console-input] @@ -50,11 +50,11 @@ EOF oc rollout status deploy/argocd-server - After a few seconds, the deployment should be complete: ++ [.bordershadow] image::05/argocd-rollout.png[] -**Optional**: connect to the ArgoCD UI -If you know and understand ArgoCD and want to look at its interface ... +**Optional**: If you know and understand ArgoCD and want to look at its interface, look at the following details. Otherwise, you can skip to the next step. [%collapsible] ==== Now that Argo is deployed, you can connect to its UI through its Route. @@ -74,7 +74,7 @@ image::05/argocd-route.png[] - You can now connect to the ArgoCD UI using your given credentials ({user}/{password}). ==== -== Deploy the applications via GitOps +== Deploy the application(s) via GitOps Now that ArgoCD is deployed in your namespace, we can use it to deploy the application via GitOps. @@ -105,7 +105,6 @@ spec: syncOptions: [CreateNamespace=false] EOF - - Once the application is deployed, you can check its Route with the following command: + [.lines_space] @@ -115,4 +114,4 @@ EOF [source, text] echo " Application URL : https://$(oc get route ic-app -ojsonpath='{.status.ingress[0].host}')/ " -Open the URL in your browser to access the application and head for the next step. +Open the URL in your browser to access the application and head to the next step. diff --git a/content/modules/ROOT/pages/05-04-web-app-validating.adoc b/content/modules/ROOT/pages/05-04-web-app-validating.adoc index d87123ef..41522a56 100644 --- a/content/modules/ROOT/pages/05-04-web-app-validating.adoc +++ b/content/modules/ROOT/pages/05-04-web-app-validating.adoc @@ -10,9 +10,9 @@ image::05/original-app-1.png[] [.bordershadow] image::05/original-app-2.png[] -== New App +== Prototype App -The second item menu on the left, New App, gives a view of what the new application looks like. You can navigate each claim and see that the interface is much more user friendly, with a nice interface to navigate the claim and see the pictures that were attached to it. +The second item menu on the left, New App, gives a view of what the prototype of the new application looks like. You can navigate each claim and see that the interface is much more user friendly, with a nice interface to navigate the claim and see the pictures that were attached to it. Moreover, for the claims that have been processed, you can see the summary of this claim as well as the customer sentiment analysis, the location and time of the event that have been extracted from the claim, and the pictures that have been processed by the AI. diff --git a/content/modules/ROOT/pages/06-01-potential-imp-ref.adoc b/content/modules/ROOT/pages/06-01-potential-imp-ref.adoc index 1fb057c3..5303e421 100644 --- a/content/modules/ROOT/pages/06-01-potential-imp-ref.adoc +++ b/content/modules/ROOT/pages/06-01-potential-imp-ref.adoc @@ -3,18 +3,18 @@ include::_attributes.adoc[] == To the lab materials -Although we welcome feedback, critique, we prefer https://github.com/rh-aiservices-bu/insurance-claim-processing/issues[Issues,window=_blank] and https://github.com/rh-aiservices-bu/insurance-claim-processing/pulls[Pull Requests,window=_blank]. +If you have any feedback regarding this Lab, please use https://github.com/rh-aiservices-bu/insurance-claim-processing/issues[Issues,window=_blank] and https://github.com/rh-aiservices-bu/insurance-claim-processing/pulls[Pull Requests,window=_blank]. But this section of the lab is not meant for Improvements and Refinements **to the lab**. Instead it is about... == To the presented prototype -What we have showed in this lab is a very rough prototype, put together very quickly, in order to demonstrate: +What we have shown in this lab is a very rough prototype, put together very quickly, in order to demonstrate: -* what could be done -* that it was do-able at all +* which improvements could be done +* how long it would take to do them -In such a situation, it's common to go fast and make short term decisions since there is no guarantee that this will become a real project. +In such a situation, it is common to go fast and make short-term decisions since there is no guarantee that this will become a real project. Therefore, now, we need to review what was done, and make suggestions along these axes. @@ -22,9 +22,9 @@ Therefore, now, we need to review what was done, and make suggestions along thes * Is our current application robust enough? Describe what scenarios might make it harder to use? -* How efficiently are we going to be able to make updates to its components if we need to? Is there a chance that making a change to the LLM model, the YOLOV8 model, the database, the application, we introduce breaking changes? +* How efficiently are we going to be able to make updates to its components if we need to? Is there a chance that making a change to the LLM model, the YOLOV8 model, the database, the application, will introduce breaking changes? -* If that is the case, what would we want to do to avoid those possibilities? +* If that is the case, what would we want to do to avoid those risks? * Which parts seem to still be manual and time-consuming, and how could we automate them better, to avoid human errors if we can. @@ -38,14 +38,14 @@ If you want to read what **we** thought could be improved, read below! (response ** not the same make or color car as what is on file ** mismatch in license plate, if visible in the picture * We've only scratched the surface with gitops and pipelines here -** There was not performance testing done. If too many users connect at the same time, it might overwhelm either the app, the database, the LLM, etc... +** There was no performance testing done. If too many users connect at the same time, it might overwhelm either the app, the database, the LLM, etc... * Currently, most simple changes would probably end up breaking the application. And the person who, for example decides to change Mistral7B for Flan-T5-Small would not necessarily realize that. ** It would be critical to have multiple instances (Dev/Test/UAT/Prod) of the application ** It would also be required to have integration pipelines run in these environments to confirm that changes made do not break the overall application. * We could ask the LLM to start writing a response to the customer. ** It could be just to ask for missing details. ** or it could be to let them know whether the claim is accepted or denied -* However, to do this, the LLM would have to be aware of the policies that the insurance company has to make those determinations. +* However, to do this, the LLM would have to be aware of the policies that the insurance company uses to make those determinations. ** This could be an interesting use-case for the https://research.ibm.com/blog/retrieval-augmented-generation-RAG[RAG,window=_blank] approach. ==== diff --git a/content/modules/ROOT/pages/07-01-end-of-lab.adoc b/content/modules/ROOT/pages/07-01-end-of-lab.adoc index ab17aa21..e12c0fe7 100644 --- a/content/modules/ROOT/pages/07-01-end-of-lab.adoc +++ b/content/modules/ROOT/pages/07-01-end-of-lab.adoc @@ -5,6 +5,8 @@ We hope that the materials we used during this time together were useful and gav If you notice https://github.com/rh-aiservices-bu/insurance-claim-processing/issues[issues,window=_blank] with the content and/or want to send us a https://github.com/rh-aiservices-bu/insurance-claim-processing/pulls[pull request,window=_blank], we'll appreciate it very much. +The instructions of this lab are always available at https://rh-aiservices-bu.github.io/insurance-claim-processing/[https://rh-aiservices-bu.github.io/insurance-claim-processing/,window=_blank]. Make note of this URL as it will be updated with new content as we continue to improve the lab. + == Acknowledgements Here is a list of people who have participated, directly or indirectly, to the creation and delivery of this lab. diff --git a/content/modules/ROOT/pages/index.adoc b/content/modules/ROOT/pages/index.adoc index aa71bad6..f7b69797 100644 --- a/content/modules/ROOT/pages/index.adoc +++ b/content/modules/ROOT/pages/index.adoc @@ -12,6 +12,14 @@ This lab will illustrate how the use of various AI/ML technologies can be combin The information, code, models and techniques it contains are illustrations of what a first prototype could look like. It is not the definitive way of addressing the stated requirements. +== Disclaimer + +This lab is an example of what a customer could build using OpenShift AI. OpenShift AI has no specific feature related to Insurance Claim Processing. + +This lab makes use of large language models (LLM) and image processing models. These models are not included in the OpenShift AI product. They are provided as a convenience for this lab. + +The quality of these models is enough for a prototype. Choosing the right model to use in a production environment is a complex task that requires a lot of experimentation and tuning. This lab does not cover this aspect. + == Timetable This is a tentative timetable for the materials that will be presented today. @@ -40,7 +48,7 @@ a|- summarization check |Image Processing |20 | Hands-On a|- car recognition checks - re-training exercise -- model deployment (automated) +- model deployment |Web App |20 | Hands-On a|- deployment