Getting Started with Git AI Library

Experiment Management

Creating an experiment

Overview

The Experiment class is designed for the creation and management of experiments. Upon instantiation, this class automatically creates a dedicated branch for the experiment and switches to this branch. All subsequent checkpoints are committed to this branch, ensuring that every stage of the experiment is captured and versioned.

Create an Experiment

To create an experiment, simply instantiate the Experiment class in .py file under your root project. This action creates a new branch and switches the working directory to this branch:

from git_ai.metrics.experiment import Experiment
 
experiment = Experiment()

The new experiment will be created with a name that follows the format: [commit_hash]-id where [commit_hash] is the hash of the current commit and id is a unique identifier.

Checkpoint Management

• Automatic Checkpointing

When used as a context manager, the Experiment class facilitates automatic checkpointing. Upon exiting the context, it creates a final checkpoint and reverts to the original branch. This feature ensures that all changes made during the experiment are saved and that the workspace is clean after the experiment concludes.

Example of using Experiment as a context manager:

with Experiment() as experiment:
    # Your experiment code here
    # A final checkpoint is automatically created upon exiting the context
 

If it is not used as a context manager, it reverts upon the script’s completion and you need to manually create a checkpoint (see below).

• Manual Checkpointing

Users also have the flexibility to manually create checkpoints at any desired point in the experiment using the Experiment.checkpoint() method. This method is useful for documenting significant milestones or states within the experiment.

Example of manual checkpointing:

experiment.checkpoint('significant_milestone')
 

Writing Metadata

TensorBoard Integration

The Experiment class includes a metadata writer that complies with the TensorBoard API. This integration allows users to log experiment metadata, such as parameters and performance metrics, making it accessible via the Experiment.writer method.

Example of writing metadata:

writer = experiment.writer
writer.add_hparams({'lr': 0.01, 'batch_size': 32}, {'hparam/accuracy': 0.95})

This capability enhances the experiment's traceability and facilitates in-depth analysis through TensorBoard's visualization tools.

Here's a full example demonstrating the extensive capabilities of experiment management and the Experiment class:

from git_ai.metrics.experiment import Experiment
 
# Setup code...
 
with Experiment() as experiment:
    writer = experiment.writer
    # Log hyperparameters
    writer.add_hparams(...)
    # Training loop
    for epoch in epoch_count:
        # Training code...
        # Log metrics for the current epoch
        writer.add_scalar(...)
        # Generate and push checkpoint
        experiment.checkpoint(f"Checkpoint {epoch}")

The Git AI library significantly enhances AI project management within the Git ecosystem. It streamlines experiment tracking, collaboration, and analysis, leading to more efficient and successful AI development outcomes.