AltarSender — Experiment Sender to Sacred

Note: AltarSender is part of the Altar monorepo. You can either download a prebuilt executable from the monorepo releases or build and run from source.

A graphical user interface (GUI) built with Python and CustomTkinter to send experiment results to a MongoDB Sacred database. Optionally upload heavy files to MinIO or a local/network file path.

Quick guide: Sacred infrastructure (at a glance)

The simplest mapping to help you fill the app:

Tip: MongoDB stores your metadata (config, results, metrics), while MinIO or your filesystem stores the large raw files.

Features

• Send experiment metadata (config, results, metrics) to MongoDB Sacred
• Upload artifacts to MongoDB (files < 25 MB)
• Upload raw data (large files) to MinIO S3 or filesystem paths
• Batch send multiple experiments with the same folder structure
• Visual mapping of experiment files to Sacred data types

Prerequisites

• Required: Python 3.x installed
• Required: MongoDB database running (locally or on a server)
• Optional: MinIO server for large file storage

Deployment instructions: See Altar Docs for setting up MongoDB and MinIO.

Setup

1. Clone or download the monorepo:

    git clone https://github.com/DreamRepo/Altar.git
    cd Altar/AltarSender
   

2. Create and activate a virtual environment (recommended):

   python -m venv .venv
   

   Windows:

   .venv\Scripts\activate
   

   Linux/macOS:

   source .venv/bin/activate
   

3. Install requirements:

   pip install -r requirements.txt
   

From Binary Release

Prebuilt executables for Windows, macOS, and Linux are attached to the monorepo’s Releases.

1) Download

• Go to the monorepo Releases page
• Under the latest release, download the Sender asset for your OS/architecture

2) Run

• Windows (PowerShell):

    .\AltarSender.exe
  

• macOS/Linux (Terminal):

    chmod +x ./AltarSender
    ./AltarSender
  

Run the app

python app.py

If you see ModuleNotFoundError: No module named 'XXXX', run:

pip install XXXX

The app window should open.

Configuration

MongoDB connection

Enter your database credentials in the app. Use the Test Connection button to verify connectivity.

MinIO connection (optional)

If uploading raw data to MinIO, enter your MinIO credentials (endpoint, access key, secret key, bucket name).

Experiment File Configuration

Select your experiment folder

Each experiment should have its own folder. The folder name becomes the experiment name.

File categories

Config

The experiment configuration (name, conditions, instrument settings, etc.).

Supported formats: JSON, CSV, Excel

JSON example:

{
    "experiment": {
        "name": "Experiment1",
        "duration": {
            "time": 3200,
            "unit": "seconds"
        }
    }
}

With “Flatten JSON” enabled, this becomes:

{
    "experiment_name": "Experiment1",
    "experiment_duration_time": 3200,
    "experiment_duration_unit": "seconds"
}

CSV/Excel format (no header, key-value pairs):

Results

Experiment result values. Same format as Config (JSON, CSV, or Excel).

Metrics

Time series data for plotting. Supported formats: CSV, Excel.

• If columns have headers, enable Column header
• If there’s an X-axis column, enable X-axis column and select it
• Select which columns to plot in the database

Raw data

Large files to upload to MinIO or a filesystem path.

• Select a single file or an entire folder
• Choose destination: local path, network drive, and/or MinIO
• Files are renamed with a hash and organized by type

Naming convention:

• video.tiff → video/[hash]_[datetime]_video.tiff (if folder name contains datetime)
• video.tiff → video/[hash]_video.tiff (otherwise)

The hash is generated from the experiment folder name (7 alphanumeric characters).

Artifacts

Small files (< 50MB) stored directly in MongoDB. Same organization as raw data. These files can be accessed directly from Omniboard.

Warning: Storing large artifacts impacts database performance.

Sending Experiments

Single experiment

1. Select the experiment folder
2. Configure file mappings
3. Click Send experiment

Batch send (multiple experiments)

For experiments with identical folder structures:

1. Select one experiment folder and configure it
2. Enable Send multiple experiments
3. All sibling folders with the same structure will be selected
4. Click Send experiment

Example structure:

Parent folder/
├── 2023-04-17_18_13_Experiment1/
│   ├── config.json
│   ├── results.csv
│   ├── metrics.xlsx
│   ├── capture.png
│   └── raw_data/
│       ├── frames.tiff
│       └── video.mp4
│
└── 2023-04-17_18_18_Experiment2/
    ├── config.json
    ├── results.csv
    ├── metrics.xlsx
    ├── capture.png
    └── raw_data/
        ├── frames.tiff
        └── video.mp4

Configure Experiment1, enable batch mode, and both experiments will be sent.

Viewing Results

Access your experiments in Omniboard:

Building Standalone Executables

You can build a standalone executable that doesn’t require Python to be installed. The executable will include all dependencies.

Prerequisites

Ensure you have the development environment set up:

# Create and activate virtual environment
python -m venv .venv

# Windows
.venv\Scripts\activate

# Linux/macOS
source .venv/bin/activate

# Install dependencies
pip install -r requirements.txt
pip install pyinstaller

Windows

# Option 1: Use the build script (recommended)
python build_exe.py

# Option 2: Use PyInstaller directly with the spec file
pyinstaller AltarSender.spec

# Option 3: Build manually
pyinstaller --name=AltarSender --onefile --windowed --noconfirm ^
    --add-data=".venv\Lib\site-packages\customtkinter;customtkinter" ^
    --hidden-import=customtkinter --hidden-import=sacred ^
    --hidden-import=sacred.observers --hidden-import=pymongo ^
    --hidden-import=pandas --hidden-import=numpy --hidden-import=openpyxl ^
    --hidden-import=boto3 --hidden-import=botocore ^
    --collect-all=customtkinter --collect-all=sacred ^
    app.py

The executable will be in the dist/ folder: dist/AltarSender.exe

Linux

# Option 1: Use the build script (recommended)
python build_exe.py

# Option 2: Use PyInstaller directly with the spec file
pyinstaller AltarSender.spec

# Option 3: Build manually
pyinstaller --name=AltarSender --onefile --windowed --noconfirm \
    --add-data=".venv/lib/python3.*/site-packages/customtkinter:customtkinter" \
    --hidden-import=customtkinter --hidden-import=sacred \
    --hidden-import=sacred.observers --hidden-import=pymongo \
    --hidden-import=pandas --hidden-import=numpy --hidden-import=openpyxl \
    --hidden-import=boto3 --hidden-import=botocore \
    --collect-all=customtkinter --collect-all=sacred \
    app.py

The executable will be in the dist/ folder: dist/AltarSender

Make it executable if needed:

chmod +x dist/AltarSender

macOS

# Option 1: Use the build script (recommended)
python build_exe.py

# Option 2: Use PyInstaller directly with the spec file
pyinstaller AltarSender.spec

# Option 3: Build manually
pyinstaller --name=AltarSender --onefile --windowed --noconfirm \
    --add-data=".venv/lib/python3.*/site-packages/customtkinter:customtkinter" \
    --hidden-import=customtkinter --hidden-import=sacred \
    --hidden-import=sacred.observers --hidden-import=pymongo \
    --hidden-import=pandas --hidden-import=numpy --hidden-import=openpyxl \
    --hidden-import=boto3 --hidden-import=botocore \
    --collect-all=customtkinter --collect-all=sacred \
    app.py

The executable will be in the dist/ folder: dist/AltarSender.app (or dist/AltarSender binary)

Note: On macOS, you may need to right-click and select “Open” the first time to bypass Gatekeeper, or run xattr -cr dist/AltarSender.app to remove quarantine attributes.

Build Output

Troubleshooting

• Missing module errors: Add the module as a --hidden-import flag
• Missing data files: Use --add-data to include necessary files
• Large executable size: This is normal (~50-100MB) due to bundled Python and dependencies
• Antivirus warnings: False positives are common with PyInstaller; whitelist the executable if needed

Automated Builds (GitHub Actions)

This monorepo may include a GitHub Actions workflow to build executables for Windows, Linux, and macOS.

Downloading the Latest Build

1. Go to the Actions tab in this repository
2. Click on the latest successful Build Executables workflow run
3. Scroll down to Artifacts and download the version for your platform:
   • AltarSender-Windows.exe — Windows executable
   • AltarSender-Linux — Linux executable
   • AltarSender-macOS — macOS executable
4. Extract the ZIP file to get the executable

Note: On Linux/macOS, you may need to make the file executable:

chmod +x AltarSender-Linux   # or AltarSender-macOS

Releases

Releases for individual components may be consolidated at the monorepo level. Prefer building from source using the instructions above.

Manual Workflow Trigger

You can also manually trigger a build:

1. Go to the Actions tab
2. Select Build Executables workflow
3. Click Run workflow → Run workflow

Related

• AltarDocker docs — Deploy MongoDB, MinIO, Omniboard, and AltarExtractor
• AltarExtractor docs — Browse and filter Sacred experiments in a web UI