Skip to content

Source code for chunking code in multiple different languages

Notifications You must be signed in to change notification settings

davidwrossiter/langchunk

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

13 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Lang chunk

I created langchunk to chunk entire codebases correctly.

RAG systems rely on data that has been properly chunked and embedded. Langchunk allows you to create high-quality chunks from any codebase including metadata about the file structure of the directory and the content of each file within it.


Current capabilities

  • Allows for chunking of javascript source code.
  • Chunks based on Function declaration, class declaration, variable declaration, and expression statements.

Steps for future versions

  • Allow for different languages
  • Generate embeddings for vector databases

About

Source code for chunking code in multiple different languages

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published