A console/terminal pipeline for training and running Transformer vision models on Windows 11 and Ubuntu Linux 24.04. Three independent models sit behind one set of launchers: ViT and Swin for image classification, plus SAM2 for image segmentation (inference only). The example datasets chest (NORMAL / PNEUMONIA) and plate are included.

 

[Models]

  • vit: image classification, backed by timm (vit_base_patch16_224).
  • swin: image classification, backed by timm (swin_base_patch4_window7_224).
  • sam2: image segmentation (inference only), backed by the vendored third_party/sam2/ source.

[Folder Structure]

  • CLAUDE.md / AGENT.md: program specification (identical content).
  • CHANGELOG.md: change log.
  • copyright.txt: copyright notice.
  • config/vit.yaml, config/sam2.yaml: the ViT and SAM2 configs.
  • data/{dataset}/: a per-dataset swin.yaml, ImageFolder train/, val/ and test/, plus model/ (trained checkpoints).
  • checkpoints/: ViT checkpoints and SAM2 base weights, downloaded by 02_venv_pip.
  • third_party/sam2/: the vendored upstream SAM2 source, installed editable into the venv.
  • outputs/: eval reports, confusion matrices and inference JSON.
  • src/common/: get_device() and the data pipeline; src/vit/, src/swin/ and src/sam2_demo/ are the model packages.
  • src/sample_images/demo.jpg: sample image for inference.
  • src/requirements.txt: the Python dependencies installed by 02_venv_pip.
  • venv/: Python virtual environment, created by 01_venv_create and removed by 99_venv_clear.

[Prerequisites]

  • Operating system: Windows 11 (the .bat launchers) or Ubuntu Linux 24.04 (the .sh launchers).
  • Python 3.10 or newer on PATH (SAM2 installs with --ignore-requires-python, so 3.9 also works).
  • git and curl on PATH for 02_venv_pip, which clones SAM2 and downloads the checkpoints (it falls back to a zip download when git is missing).
  • Optional: an NVIDIA GPU with a matching CUDA build of PyTorch. The device is selected automatically in the order cuda → mps → cpu.
  • timm downloads pretrained weights from Hugging Face on first use; set pretrained to false in the config, or set HF_ENDPOINT, if that is blocked.

[Instructions]

Step 1: Create the virtual environment

Recreate the venv and upgrade pip, setuptools and wheel.

 

Windows: src\01_venv_create.bat

Linux: bash src/01_venv_create.sh

Step 2: Install the packages

Install requirements.txt, clone and editable-install SAM2, and download the SAM2 base checkpoints.

 

Windows: src\02_venv_pip.bat

Linux: bash src/02_venv_pip.sh

Step 3: Activate the virtual environment (optional)

Activate the venv and set PYTHONPATH=src. On Linux, source the script. Steps 5, 6 and 7 activate the venv automatically, so this step is optional.

 

Windows: src\03_venv_activate.bat

Linux: source src/03_venv_activate.sh

Step 4: Verify the CIFAR-10 dataset (optional)

Verify the layout of a manually downloaded Kaggle CIFAR-10 dataset. Used by the ViT demo only.

 

Windows: src\04_download_cifar10.bat

Linux: bash src/04_download_cifar10.sh

Step 5: Train a model

Train the named model. The first argument is vit or swin; everything after it is forwarded to the program.

 

Windows: src\05_train.bat swin --config data\chest\swin.yaml

Linux: bash src/05_train.sh swin --config data/chest/swin.yaml

Step 6: Evaluate the model

Evaluate on the whole validation set and report accuracy plus a confusion matrix.

 

Windows: src\06_eval.bat swin --checkpoint data\chest\model\swin_best.pt --class-names "NORMAL,PNEUMONIA"

Linux: bash src/06_eval.sh swin --checkpoint data/chest/model/swin_best.pt --class-names "NORMAL,PNEUMONIA"

Step 7: Single-item inference

Run inference on a single image. The first argument is vit, swin or sam2.

 

Windows: src\07_inference.bat swin --image data\chest\test\NORMAL-11.jpg --checkpoint data\chest\model\swin_best.pt --class-names "NORMAL,PNEUMONIA"

Windows: src\07_inference.bat sam2 --image src\sample_images\demo.jpg --mode auto

Linux: bash src/07_inference.sh swin --image data/chest/test/NORMAL-11.jpg --checkpoint data/chest/model/swin_best.pt --class-names "NORMAL,PNEUMONIA"

Linux: bash src/07_inference.sh sam2 --image src/sample_images/demo.jpg --mode auto

Step 8: Deactivate the virtual environment

Deactivate the current venv.

 

Windows: src\98_venv_deactivate.bat

Linux: bash src/98_venv_deactivate.sh

Step 9: Remove the virtual environment

Delete the venv/ folder.

 

Windows: src\99_venv_clear.bat

Linux: bash src/99_venv_clear.sh

[Config Mapping]

The launchers for steps 5, 6 and 7 all take the model name first and forward the rest of the arguments unchanged. Each model defaults to its own config:

  • vit: config/vit.yaml
  • swin: data/chest/swin.yaml (pass --config data/plate/swin.yaml to use the plate dataset)
  • sam2: config/sam2.yaml

[Dataset Format]

  • ImageFolder layout: data/{dataset}/train/{class}/*.jpg and val/{class}/*.jpg; test/ holds loose images for inference.
  • data.name in the config selects the pipeline: imagefolder (chest, plate), fake (synthetic smoke test), cifar10 (downloaded automatically by torchvision), or kaggle_cifar10 (manual download, verified with 04_download_cifar10).
  • Checkpoints record num_classes; pass --class-names "a,b,c" at eval and inference time for readable labels.

[References]

 

This SDK is built in AppForAI - AI Dev Tools.

 

Purchase license separately: USD 600, permanent authorization, single APP authorization, single machine authorization, one-year activation, one-year download, one-year update, one-year email technical support.