Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discussion: Cortex.cpp Data Structures #1040

Closed
namchuai opened this issue Aug 28, 2024 · 15 comments
Closed

Discussion: Cortex.cpp Data Structures #1040

namchuai opened this issue Aug 28, 2024 · 15 comments
Assignees
Labels
type: planning Opening up a discussion

Comments

@namchuai
Copy link
Contributor

namchuai commented Aug 28, 2024

Overview

  • What is Cortex's data structure? (i.e. files that are persisted to disk)
  • This should be different from Cortex Platform

An unopinionated (hopefully) to store cortex's files and folders.

Couple of requirements:

  • Allow 3rd apps to be config where to store files with ease
  • Easy for cortex to grab the configurations
  • Easy for remove without interfere with user's data (user's own models, threads, files, etc.)
  • Updating cortex should not damage user's data

Discussion

  • Should we have a /cortex folder (will there be namespace collisions?)
  • Should folder be hidden? (e.g. .cortex
  • Where are we storing the config files?

Related

@imtuyethan imtuyethan moved this to Planning in Jan & Cortex Sep 2, 2024
@imtuyethan imtuyethan added the type: planning Opening up a discussion label Sep 2, 2024
@dan-menlo dan-menlo moved this from Planning to Scheduled in Jan & Cortex Sep 3, 2024
@dan-menlo dan-menlo added the needs pm Needs product level decisions label Sep 3, 2024
@dan-menlo dan-menlo assigned hiento09 and unassigned namchuai Sep 3, 2024
@dan-menlo dan-menlo moved this from Scheduled to Need Investigation in Jan & Cortex Sep 3, 2024
@dan-menlo dan-menlo changed the title Discussion: Cortex data folder Discussion: Cortex Data Structures Sep 4, 2024
@dan-menlo dan-menlo changed the title Discussion: Cortex Data Structures Discussion: Cortex.cpp Data Structures Sep 4, 2024
@vansangpfiev
Copy link
Contributor

vansangpfiev commented Sep 4, 2024

I think we can separate application binary and data:

  • Execution file: contains only cortex binary
    Windows: C:\Users\<username>\AppData\Local\cortex
    MacOS: /usr/local/bin/cortex
    Ubuntu: /usr/bin/cortex
  • Application data:
    Windows: C:\Users\<username>\.cortex
    MacOS: ~/.cortex
    Ubuntu: ~/.cortex

We have different folders to store different application data builds:

~/.cortex
  |___ models
  |___ logs
  |___ engines
          |___ cortex.llamacpp

~/.cortex-beta

~/.cortex-nightly

We also have an application config file which allow user to change the data path and other parameters with the default values mentioned above.
https://github.com/janhq/cortex/issues/1031
cc: @dan-homebrew @namchuai @hiento09

@hiento09
Copy link
Contributor

hiento09 commented Sep 4, 2024

I put my comment here https://github.com/janhq/cortex/issues/1039#issuecomment-2327901258 cc @dan-homebrew @namchuai @vansangpfiev .
I understand that each channel (stable, beta, nightly) is considered a separate app, and we allow users to install them in parallel. Therefore, the config files should also be kept separate.

(Ex for windows:

  • Stable: C:\Users<username>\AppData\Local\cortex\cortex-cpp.exe
  • Beta: C:\Users<username>\AppData\Local\cortex-beta\cortex-cpp-beta.exe
  • Nighty: C:\Users<username>\AppData\Local\cortex-nightly\cortex-cpp-nightly.exe
    )

@freelerobot
Copy link
Contributor

freelerobot commented Sep 4, 2024

@vansangpfiev - this is well thought through.

Questions:

  1. Does this mean we will also have the following:
~/.cortex-platform
~/.jan

cc @louis-jan @marknguyen1302 for context when thinking through Cortex Platform data folders

So for a user who uses Jan, the most top level application, altogether do they have:

~/.jan
~/.cortex-platform
~/.cortex
  1. For naming, I think @dan-homebrew was keen to establish cortexcpp, which is clearer naming
    So is it instead: ~/.cortexcpp

  2. Can we break down the following folders more

@dan-menlo
Copy link
Contributor

dan-menlo commented Sep 4, 2024

@0xSage Let's scope this discussion to cortex:

A few points:

  • I think we should use cortex.cpp to refer to Cortex (easier to SEO)
  • The folder can then be cortex.cpp (reduces chance for edge case where user already has a /cortex)

Additionally, should this folder be hidden? (e.g. ~/.cortex.cpp or ~/cortex.cpp)?

@vansangpfiev
Copy link
Contributor

I agree, we can use cortex.cpp instead of cortex. And I think the data folder should be hidden ~/.cortex.cpp (like .ssh)
cc: @dan-homebrew @hiento09

@freelerobot
Copy link
Contributor

freelerobot commented Sep 4, 2024

Should we put configuration file .cortexrc inside the data folder .cortex.cpp, i.e related data together, making it easier for users to find and manage their configuration files.

It's weird if cortex config is in ~, but engine configs are in .cortex.cpp

@dan-menlo
Copy link
Contributor

dan-menlo commented Sep 4, 2024 via email

@hiento09
Copy link
Contributor

hiento09 commented Sep 4, 2024

Should we put configuration file .cortexrc inside the data folder .cortex.cpp, i.e related data together, making it easier for users to find and manage their configuration files.

It's weird if cortex config is in ~, but engine configs are in .cortex.cpp

The purpose of the .cortexrc file is to customize configurations such as the data folder, port, listen address, etc. For example, the .cortex.cpp folder is the app data folder specified in the .cortexrc file, and its default value is ~/.cortex.cpp.
Ref https://github.com/janhq/cortex/issues/1031

@freelerobot
Copy link
Contributor

Nit: naming a data folder: ~/.cortex.cpp might be confusing for users. Some users might expect a .cpp file extension to represent a C++ source file, not a folder.

And should we be consistent across package names & folder names, e.g.

Option 1
brew install cortexcpp
~/.cortexcpp

Option 2
brew install cortex-cpp
~/.cortex-cpp

Sorry to nitpick, but this is one of those things where changing in the future is a pain.

@hiento09
Copy link
Contributor

hiento09 commented Sep 4, 2024

Yep, I completely agree that the app/package name and the app data folder should be consistent with each other to avoid confusing the user. Now we just need to finalize the name.

@hiento09
Copy link
Contributor

hiento09 commented Sep 4, 2024

@0xSage Let's scope this discussion to cortex:

A few points:

  • I think we should use cortex.cpp to refer to Cortex (easier to SEO)
  • The folder can then be cortex.cpp (reduces chance for edge case where user already has a /cortex)

Additionally, should this folder be hidden? (e.g. ~/.cortex.cpp or ~/cortex.cpp)?

@dan-homebrew , the naming of the folder and the binary file is up to us. However, I searched and found that no one uses a dot in their package name or app name. You can refer to it here: https://github.com/Homebrew/homebrew-core/tree/master/Formula/c. I think we should consider @0xSage 's options 1 and 2 above. Including a character that violates naming rules for package names or app names on other platforms (brew, npm) could cause us significant difficulties later on.

@freelerobot
Copy link
Contributor

Cool, I suggest just cortexcpp then?

From a product perspective:

  1. Official name CortexCPP
  2. package names cortexcpp
  3. folder names cortexcpp

@namchuai
Copy link
Contributor Author

namchuai commented Sep 4, 2024

If I have one vote then I will vote for option 1

@dan-menlo
Copy link
Contributor

dan-menlo commented Sep 4, 2024

@namchuai @vansangpfiev @nguyenhoangthuan99 @hiento09 Did we make a decision on whether it should be a hidden folder, or a visible folder?

Referring to my post in #1092: Folder name should be ~/.cortexcpp

Cortex Model Folder Visibility

Should it be a visible folder instead? i.e. ~/cortexcpp

Application data:

  • Windows: C:\Users<username>\cortexcpp
  • MacOS: ~/cortexcpp
  • Ubuntu: ~/cortexcpp

We will have different folders for nightly, i.e:

  • ~/cortexcpp-beta
  • ~/cortexcpp-nightly

Config File

  • ~/.cortexrc

Execution file (Cortex Binary)

  • contains only cortex binary
  • Windows: C:\Users<username>\AppData\Local\cortex
  • MacOS: /usr/local/bin/cortex
  • Ubuntu: /usr/bin/cortex

@dan-menlo
Copy link
Contributor

dan-menlo commented Sep 4, 2024

I've also updated the naming structure of Cortex files that the Installer + Uninstaller will track:

#1039 (comment)

Installer:

  • Windows
    • Format: .exe
    • Binary location:
      • Stable: C:\Users\<username>\AppData\Local\cortexcpp\cortex.exe
      • Beta: C:\Users\<username>\AppData\Local\cortexcpp-beta\cortex-beta.exe
      • Nighty: C:\Users\<username>\AppData\Local\cortexcpp-nightly\cortex-nightly.exe
    • Configuration file:
      • Stable: C:\Users\<username>\.cortexrc
      • Beta: C:\Users\<username>\.cortexrc-beta
      • Nighty: C:\Users\<username>\.cortexrc-nightly
    • Data folder (contains engines, models, logs)
      • Stable: C:\Users\<username>\.cortexcpp
      • Beta: C:\Users\<username>\.cortexcpp-beta
      • Nighty: C:\Users\<username>\.cortexcpp-nightly
  • Linux:
    • Format: .deb
    • Binary location:
      • Stable: /usr/bin/cortexcpp
      • Beta: /usr/bin/cortexcpp-beta
      • Nighty: /usr/bin/cortexcpp-nightly
    • Configuration file:
      • Stable: /home/<username>/.cortexrc
      • Beta: /home/<username>/.cortexrc-beta
      • Nighty: /home/<username>/.cortexrc-nightly
    • Data folder (contains engines, models, logs)
      • Stable: /home/<username>/.cortexcpp
      • Beta: /home/<username>/.cortexcpp-beta
      • Nighty: /home/<username>/.cortexcpp-nightly
  • Macos:
    • Format: .pkg
    • Binary location:
      • Stable: /usr/local/bin/cortexcpp
      • Beta: /usr/local/bin/cortexcpp-beta
      • Nighty: /usr/local/bin/cortexcpp-nightly
    • Configuration file:
      • Stable: /Users/<username>/.cortexrc
      • Beta: /Users/<username>/.cortexrc-beta
      • Nighty: /Users/<username>/.cortexrc-nightly
    • Data folder (contains engines, models, logs)
      • Stable: /Users/<username>/.cortexcpp
      • Beta: /Users/<username>/.cortexcpp-beta
      • Nighty: /Users/<username>/.cortexcpp-nightly

@janhq janhq locked and limited conversation to collaborators Sep 5, 2024
@dan-menlo dan-menlo converted this issue into discussion #1115 Sep 5, 2024
@github-project-automation github-project-automation bot moved this from Need Investigation to Completed in Jan & Cortex Sep 5, 2024
@dan-menlo dan-menlo removed the needs pm Needs product level decisions label Sep 6, 2024
@dan-menlo dan-menlo moved this from Completed to Discontinued in Jan & Cortex Sep 6, 2024

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Labels
type: planning Opening up a discussion
Projects
Archived in project
Development

No branches or pull requests

6 participants