Console/terminal pipeline for running and fine-tuning SAM2 (Segment Anything Model 2), supporting Windows 11 and Ubuntu Linux 24.04. SAM2 is a prompt-based segmentation model: you give it a point, a set of points or a box, and it cuts out the matching object. Inference supports four prompt modes; fine-tuning freezes the image encoder and trains only the prompt encoder and the mask decoder, using binary masks as labels. Two example datasets ship with the project: CT (liver) and nucleus. The device is auto-selected in the order cuda, mps, cpu.

 

[Folder Structure]

  • CLAUDE.md / AGENT.md: program specification (identical content).
  • CHANGELOG.md: change log.
  • copyright.txt: copyright notice.
  • config/sam2.yaml: the default inference config (base checkpoint, model cfg, output dir).
  • data/{dataset}/: the per-dataset sam2.yaml, plus train/ (image and gt), val/, test/ and model/ (fine-tuned checkpoints).
  • checkpoints/: the base SAM2 weights, downloaded by step 2.
  • third_party/sam2/: the vendored upstream SAM2 source, installed editable into the venv.
  • outputs/sam2/: the default inference output folder, holding *_sam2.png and *_sam2.json.
  • src/common/: get_device(); src/sam2_demo/: SAM2Wrapper, the dataset, training and inference code.
  • src/requirements.txt: the Python dependencies installed by step 2.
  • venv/: Python virtual environment, created by step 1 and removed by 99_venv_clear.

[Prerequisites]

  • Operating system: Windows 11 (the .bat launchers) or Ubuntu Linux 24.04 (the .sh launchers).
  • Python 3.10 or newer on PATH. SAM2 installs with --ignore-requires-python, so 3.9 also works.
  • git and curl on PATH for step 2, which clones SAM2 and downloads the checkpoints; it falls back to a zip download when git is missing.
  • Optional: an NVIDIA GPU with a matching CUDA build of PyTorch. The device is auto-selected in the order cuda, mps, cpu.

[Instructions]

Step 1: create the virtual environment

Recreate the venv and upgrade pip, setuptools and wheel.

 

Windows 11: src\01_venv_create.bat

Ubuntu 24.04: bash src/01_venv_create.sh

 

Step 2: install the dependencies and the weights

Install requirements.txt, clone and editable-install SAM2, and download the base checkpoints. This step needs a network connection.

 

Windows 11: src\02_venv_pip.bat

Ubuntu 24.04: bash src/02_venv_pip.sh

 

Step 3: activate the virtual environment (optional)

Activate the venv and set PYTHONPATH=src. On Linux, source it. Steps 4 and 5 activate the venv themselves, so this step is optional.

 

Windows 11: src\03_venv_activate.bat

Ubuntu 24.04: source ./src/03_venv_activate.sh

 

Step 4: fine-tune the model

Fine-tune SAM2 on a dataset. The dataset name follows the command, and you can add options such as --epochs and --lr.

 

Windows 11: src\04_train.bat CT --epochs 10

Ubuntu 24.04: bash src/04_train.sh CT --epochs 10

 

Step 5: run inference

Segment a single image, choosing the prompt mode and the prompt itself.

 

Windows 11: src\05_inference.bat --image data\CT\test\image\liver-1.png --mode box --box "50,80,400,360"

Ubuntu 24.04: bash src/05_inference.sh --image data/CT/test/image/liver-1.png --mode box --box "50,80,400,360"

 

Step 6: deactivate and clear the virtual environment (optional)

98_venv_deactivate leaves the venv; 99_venv_clear deletes the venv/ folder to free disk space when you no longer need it.

 

Windows 11: src\98_venv_deactivate.bat or src\99_venv_clear.bat

Ubuntu 24.04: bash src/98_venv_deactivate.sh or bash src/99_venv_clear.sh

 

[Inference Modes]

  • auto: automatic "everything" masks, keeping the top K by area (--top-k-masks). Weak on complex backgrounds.
  • each-point: one mask per point, which suits scenes with several objects. Pass --points "x1,y1;x2,y2;...".
  • points: all the points act as prompts for a single object.
  • box: segment inside a rectangle. Pass --box "x1,y1,x2,y2".
  • For an input demo.jpg the outputs are outputs/sam2/demo_sam2.png (overlay, bounding box and score) and outputs/sam2/demo_sam2.json (per-mask metadata). Override the output folder with --output-dir or in config/sam2.yaml.

[Fine-Tune Data Format]

  • data/{dataset}/sam2.yaml points to train/image and train/gt, and to val/image and val/gt.
  • Every image needs a same-named binary PNG mask in the matching gt/ folder; pixels above mask_threshold (127 by default) count as foreground.
  • A bounding box is derived from each ground-truth mask and used as the SAM2 box prompt. The image encoder is frozen, so only the prompt encoder and the mask decoder train.
  • The checkpoint with the best IoU is saved as sam2_finetuned.pt.

[References]

 

This SDK is built in AppForAI - AI Dev Tools.

 

Purchase license separately:  USD 600, permanent authorization, single APP authorization, single machine authorization, one-year activation, one-year download, one-year update, one-year email technical support.