Creating sample packages

[ghost]

Is your feature request related to a problem? Please describe.

Just like the sample packages that come with OSD, I'd like to create my own sample package.

Describe the solution you'd like

A way to implement a sample package with sample data, where timestamp fields are replenished with current times.

Describe alternatives you've considered

¯_(ツ)_/¯

I use a script atm but would be great for demo purposes to have this inbuilt.

Additional context

[ahopp]

Thanks @BT-93! I like the idea of making this more accessible. If nothing else, I can imagine a use case where plugins would want to have sample packages/sample data for the specific plugin use case (e.g., the observability plugin would probably like to have bespoke sample data to best highlight their capabilities/features) in a more accessible manner. @BT-93 were you thinking about a solution locally (e.g., for use in just your instance) or did you want to see a solution where you can add sample packages to the distro or share with the community?

The latter seems like the ideal future state, but we would have to think through how someone in the community could easily create a sample packages/data (implementation) and then how the community would access them or include them in their instance (access/sharing model). We don't want to bloat the experience of course, but I could see a world where there is a catalog of available samples across use case, plugin, etc.

[ahopp]

@kgcreative @kavilla @ashwin-pc I think we've talked about similar ideas in the past - do you have any thoughts?

[ashwin-pc]

@BT-93 I'm not entirely sure i understand the ask here. Are you referring to plugins here? If thats the case we already have a package osd-plugin-generator to do this. It can scaffold 3 types of plugins:

• An Example plugin that resides within /examples
• A Core plugin that resides within /src/plugins
• Or an external plugin that isnt committed into the codebase and resides within /plugins. I assume this is what you want to do.

Loading a plugin dynamically from the Dashboard is not a feature we have today but is being discussed in #287.

As for loading sample data, even the existing sample data feature simple performs a bulk upload to upload sample data. This feature can be mimicked easily using the existing data plugin once your sample plugin is scaffolded. The example plugins are a good way to see how to build such a plugin.

[ghost]

@ahopp I too am in favour of the latter you mentioned. I don't have a solution locally I've been digging around to try and work out how the samples are created currently. My script just sends random data with relative timestamps between dates to OS.

@ashwin-pc
I see, thanks for the info. I'm just interested in loading sample data. I haven't had much time to dig around to see how the sample data feature fully works. I'll have a look at the sample plugins. I did notice there was a script that replenishes timestamps in there.

My request was a bit vague. I'm interested in how it currently works so I can mimic something in the meantime. I'm also interested in a way to create community sample packages containing data, visuals, etc.

[joshuarrrr]

While this is a different intended use case, it has some overlap with basic UI support for importing data: #1791

[ashwin-pc]

@BT-93 Thanks for the clarification. We can go many different ways to achieve what you are looking for. From a project perspective the most useful feature here would be as @joshuarrrr mentioned a plugin that can import static data. This wont address your concern for timestamps but it should make loading data a lot easier for testing purposes.

for your specific issue here, I can reccomend two approaches.

1. use logstash to generate periodic data and upload to opensearch.
2. create a sample data generator plugin that can take in some parameters and generate periodic data.

Installing sample data is handled in https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/plugins/home/server/services/sample_data/routes/install.ts#L0-L1 which you can reference for your use case if you decide to go with the second approach.

[ahopp]

@BT-93 Does @ashwin-pc comment make sense? Anything else you need?

[ghost]

@ahopp Yes it does, thanks! All good. Thanks all for the info.

[seraphjiang]

@Flyingliuhub this might align with what playground and create generic way to support ingestion example for user e.g. demo for plugin.