indexion segment Split text into contextual segments using divergence-based, TF-IDF, or punctuation strategies. When to Use User needs to chunk text for RAG or embedding pipelines User wants to split a document into meaningful sections User asks to segment text for processing Preparing text for similarity analysis at sub-document level Usage

Default window divergence strategy

indexion segment < input-file

< output-dir

TF-IDF based segmentation

indexion segment --strategy = tfidf < input-file

< output-dir

Show more