Console/terminal demo of Chinese NLP with the SnowNLP library, supporting Windows 11 and Ubuntu Linux 24.04. The main feature is sentiment analysis: the program returns a score between 0 and 1, where a value near 1 means positive and a value near 0 means negative. A trained sentiment model ships with the project, and you can also retrain it on your own corpus. The demo also covers the library's other features: word segmentation, part-of-speech tags, pinyin, Traditional to Simplified conversion, sentence splitting, keywords, summarization, tf, idf and similarity. Everything runs offline in pure Python with no GPU.

 

[Folder Structure]

  • CLAUDE.md / AGENT.md: program specification (identical content).
  • CHANGELOG.md: change log.
  • copyright.txt: copyright notice.
  • data/examples/: the notebooks' Chinese example inputs, as txt and json.
  • data/sentiment_train/: the training corpus, neg.txt (18,576 lines) and pos.txt (16,548 lines).
  • model/sentiment/: the shipped sentiment model, which is never overwritten.
  • model/train/: the models produced by step 6.
  • tool/tools.py: shared helpers.
  • src/: the scripts, the launcher scripts and this document.
  • venv/: Python virtual environment, created by step 1.

[Prerequisites]

  • Operating system: Windows 11 or Ubuntu Linux 24.04.
  • Python 3 and pip.
  • The dependency (snownlp) is listed in src/requirements.txt and installed by step 1.
  • No GPU required.

[Instructions]

Step 1: create the virtual environment

Delete the old venv/, create a fresh one and install src/requirements.txt (snownlp). Aborts loudly if pip fails.

 

Windows 11: src\01_venv_create.bat

Ubuntu 24.04: bash src/01_venv_create.sh

 

Step 2: activate the virtual environment

Activate the venv you just created. On Linux, source it so the environment variables apply to the current shell.

 

Windows 11: src\02_venv_activate.bat

Ubuntu 24.04: source src/02_venv_activate.sh

 

Step 3: score one text

Score a whole text in one go. The bundled example scores around 0.98, which is positive.

 

Windows 11: src\03_sentiment.bat

Ubuntu 24.04: bash src/03_sentiment.sh

 

Step 4: score sentence by sentence

Split a multi-sentence text and score each sentence separately, which shows you which sentence is positive and which is negative.

 

Windows 11: src\04_sentiment_sentences.bat

Ubuntu 24.04: bash src/04_sentiment_sentences.sh

 

Step 5: try the other features

Choose a feature with --op: words (segmentation), tags (part of speech), pinyin, han (Traditional to Simplified), sentences (splitting), keywords, summary, tf, idf and sim (similarity).

 

Windows 11: src\06_snownlp_demo.bat --op keywords

Ubuntu 24.04: bash src/06_snownlp_demo.sh --op keywords

 

Step 6: train your own sentiment model (optional)

Retrain the sentiment model from the corpus in data/sentiment_train/. It takes about two and a half minutes and writes to model/train/; the shipped model is left untouched. Pass --model afterwards to use the new one.

 

Windows 11: src\05_train.bat

Ubuntu 24.04: bash src/05_train.sh

Use the new model: bash src/03_sentiment.sh --model model/train/sentiment.marshal

 

Step 7: clear the virtual environment (optional)

Delete the venv/ folder to free disk space when you no longer need it.

 

Windows 11: src\99_venv_clear.bat

Ubuntu 24.04: bash src/99_venv_clear.sh

 

[Python Entry Points]

  • src/sentiment_classify.py: [--text T] [--file F] [--split] [--model model/sentiment/sentiment.marshal]. Scores the whole text, or each sentence when --split is given.
  • src/train.py: [--neg data/sentiment_train/neg.txt] [--pos data/sentiment_train/pos.txt] [--output model/train/sentiment.marshal]. The corpus is one sentence per line; under Python 3 the file actually written gets a .3 suffix, that is {output}.3.
  • src/snownlp_demo.py: --op NAME [--text T] [--file F] [--count 3] [--corpus data/examples/corpus.json] [--query W]. Each --op corresponds to one of the original feature notebooks.

[Usage Notes]

  • The shipped model was trained on shopping-review sentences, so the scores fit product-review-style text best. For another domain, retrain on your own files with one sentence per line.
  • The upstream tf.ipynb and idf.ipynb notebooks had their outputs swapped; here --op tf and --op idf both do what their names say.

[References]

 

This SDK is built in AppForAI - AI Dev Tools.

 

Purchase license separately:  USD 600, permanent authorization, single APP authorization, single machine authorization, one-year activation, one-year download, one-year update, one-year email technical support.