Rate this Page

★ ★ ★ ★ ★

Packaging & Deploying#

Part 4 of 5 in the OpenEnv Getting Started Series

This guide walks you through creating a custom environment using the OpenEnv framework and the openenv CLI.

The CLI handles scaffolding, builds, validation, and deployment so you can stay focused on environment logic.

Note

New to OpenEnv? If you’re just getting started, we recommend completing the Getting Started tutorials first. They provide a conceptual introduction to OpenEnv and reinforcement learning fundamentals. This guide is for developers ready to build production-quality environments.

Quick Reference Card#

Already familiar with OpenEnv? Here’s the 8-step process at a glance:

Step	Command / Action	Description
1	`openenv init my_env`	Scaffold new environment
2	Edit `models.py`	Define Action & Observation dataclasses
3	Edit `server/my_environment.py`	Implement `reset()` and `step()` methods
4	Edit `client.py`	Implement `_step_payload()`, `_parse_result()`, `_parse_state()`
5	`openenv serve`	Start local dev server for testing
6	`openenv validate`	Validate environment structure
7	`openenv push`	Deploy to Hugging Face Hub
8	Share the URL!	Others use via `MyEnv.from_hub("you/my-env")`

CLI Quick Reference#

Command	Description
`openenv init NAME`	Scaffold new environment
`openenv serve`	Start local dev server
`openenv build`	Build Docker image
`openenv validate --verbose`	Validate environment structure
`openenv push`	Deploy to Hugging Face Hub
`openenv push --repo-id NAME`	Deploy to specific repo
`openenv push --private`	Deploy as private environment
`openenv push --registry ghcr.io/ORG`	Push to GitHub Container Registry

Tip

For a hands-on tutorial that builds a complete environment step-by-step, see Building & Sharing Environments in the Getting Started series.

Overview#

A typical workflow looks like:

Scaffold a new environment with openenv init.
Customize your models, environment logic, and FastAPI server.
Implement a typed EnvClient (WebSocket-based for persistent sessions).
Configure dependencies and the Dockerfile once.
Use the CLI (openenv build, openenv validate, openenv push) to package and share your work.

Note

These integrations are handled automatically by the `openenv` CLI when you run `openenv init`.

Prerequisites#

Python 3.11+ and uv for dependency locking
Docker Desktop / Docker Engine
The OpenEnv library installed: pip install https://github.com/meta-pytorch/OpenEnv.git

Step-by-Step Guide#

Let’s walk through the process of building a custom environment with OpenEnv.

1. Scaffold with `openenv init`#

# Run from anywhere – defaults to current directory
openenv init my_env

# Optionally choose an output directory
openenv init my_env --output-dir /Users/you/envs

The command creates a fully-typed template with openenv.yaml, pyproject.toml, uv.lock, Docker assets, and stub implementations. If you’re working inside this repo, move the generated folder under envs/.

Typical layout:

my_env/
├── __init__.py
├── README.md
├── client.py
├── models.py
├── openenv.yaml
├── pyproject.toml
├── uv.lock
└── server/
    ├── __init__.py
    ├── app.py
    ├── my_environment.py
    ├── requirements.txt
    └── Dockerfile

Python classes are generated for the action, observation, environment, and client. For example, you will find MyEnvironment, MyAction, MyObservation, and MyEnv (client) in the my_env directory based on the name you provided. The environment uses the core State class from openenv.core.env_server.types.

2. Define Models#

Edit models.py to describe your action and observation using Pydantic:

# models.py
from pydantic import Field
from openenv.core.env_server.types import Action, Observation

class MyAction(Action):
    """Your custom action."""
    command: str = Field(..., description="Command to execute")
    parameters: dict = Field(default_factory=dict, description="Command parameters")

class MyObservation(Observation):
    """Your custom observation."""
    result: str = Field(..., description="Result of the action")
    success: bool = Field(..., description="Whether the action succeeded")

3. Implement Environment Logic#

Customize server/my_environment.py by extending Environment:

# server/my_environment.py
from uuid import uuid4
from openenv.core.env_server.interfaces import Environment
from openenv.core.env_server.types import State
from models import MyAction, MyObservation

class MyEnvironment(Environment):
    def __init__(self):
        self._state = State(episode_id=str(uuid4()), step_count=0)

    def reset(self) -> MyObservation:
        self._state = State(episode_id=str(uuid4()), step_count=0)
        return MyObservation(result="Ready", success=True, done=False, reward=0.0)

    def step(self, action: MyAction) -> MyObservation:
        # Implement your logic here
        self._state.step_count += 1
        result = self._execute_command(action.command)
        return MyObservation(result=result, success=True, done=False, reward=1.0)

    @property
    def state(self) -> State:
        return self._state

4. Create the FastAPI Server#

server/app.py should expose the environment through create_app.

Important: You must pass a class or factory function (not an instance) to enable WebSocket-based concurrent sessions:

# server/app.py
from openenv.core.env_server import create_app
from ..models import MyAction, MyObservation
from .my_environment import MyEnvironment

# Pass the class (factory) - each WebSocket session gets its own instance
app = create_app(MyEnvironment, MyAction, MyObservation, env_name="my_env")

For environments with constructor arguments, create a factory function:

# server/app.py
import os
from openenv.core.env_server import create_app
from ..models import MyAction, MyObservation
from .my_environment import MyEnvironment

# Read config from environment variables
api_key = os.getenv("MY_API_KEY")
timeout = int(os.getenv("MY_TIMEOUT", "30"))

def create_my_environment():
    """Factory function that creates MyEnvironment with config."""
    return MyEnvironment(api_key=api_key, timeout=timeout)

# Pass the factory function
app = create_app(create_my_environment, MyAction, MyObservation, env_name="my_env")

5. Implement the Client#

client.py extends EnvClient so users can interact with your server via WebSocket for persistent sessions:

# client.py
from openenv.core.env_client import EnvClient
from openenv.core.client_types import StepResult
from .models import MyAction, MyObservation, MyState

class MyEnv(EnvClient[MyAction, MyObservation, MyState]):
    def _step_payload(self, action: MyAction) -> dict:
        return {"command": action.command, "parameters": action.parameters}

    def _parse_result(self, payload: dict) -> StepResult[MyObservation]:
        obs_data = payload.get("observation", {})
        obs = MyObservation(
            result=obs_data.get("result", ""),
            success=obs_data.get("success", False),
            done=payload.get("done", False),
            reward=payload.get("reward"),
        )
        return StepResult(
            observation=obs,
            reward=payload.get("reward"),
            done=payload.get("done", False),
        )

    def _parse_state(self, payload: dict) -> State:
        return State(
            episode_id=payload.get("episode_id"),
            step_count=payload.get("step_count", 0),
        )

The EnvClient maintains a persistent WebSocket connection to the server, enabling efficient multi-step interactions with lower latency compared to HTTP. Each client instance gets its own dedicated environment session on the server.

6. Configure Dependencies & Dockerfile#

The CLI template ships with pyproject.toml and server/Dockerfile. You should manage your python dependencies with uv or pip in the pyproject.toml file. Other dependencies should be installed in the Dockerfile.

Keep building from the openenv-base image so shared tooling stays available:

Dockerfile

# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the
# LICENSE file in the root directory of this source tree.

# Multi-stage build using openenv-base
# This Dockerfile is flexible and works for both:
# - In-repo environments (with local src/core)
# - Standalone environments (with openenv from pip)
# The build script (openenv build) handles context detection and sets appropriate build args.

ARG BASE_IMAGE=openenv-base:latest
FROM ${BASE_IMAGE} AS builder

WORKDIR /app

# Build argument to control whether we're building standalone or in-repo
ARG BUILD_MODE=in-repo
ARG ENV_NAME=__ENV_NAME__

# Copy environment code (always at root of build context)
COPY . /app/env

# For in-repo builds, openenv is already in the pyproject.toml dependencies
# For standalone builds, openenv will be installed from pip via pyproject.toml
WORKDIR /app/env

# Install dependencies using uv sync
# If uv.lock exists, use it; otherwise resolve on the fly
RUN --mount=type=cache,target=/root/.cache/uv \
    if [ -f uv.lock ]; then \
        uv sync --frozen --no-install-project --no-editable; \
    else \
        uv sync --no-install-project --no-editable; \
    fi

RUN --mount=type=cache,target=/root/.cache/uv \
    if [ -f uv.lock ]; then \
        uv sync --frozen --no-editable; \
    else \
        uv sync --no-editable; \
    fi

# Final runtime stage
FROM ${BASE_IMAGE}

WORKDIR /app

# Copy the virtual environment from builder
COPY --from=builder /app/env/.venv /app/.venv

# Copy the environment code
COPY --from=builder /app/env /app/env

# Set PATH to use the virtual environment
ENV PATH="/app/.venv/bin:$PATH"

# Set PYTHONPATH so imports work correctly
ENV PYTHONPATH="/app/env:$PYTHONPATH"

# Health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
    CMD curl -f http://localhost:8000/health || exit 1

# Run the FastAPI server
# The module path is constructed to work with the /app/env structure
CMD ["sh", "-c", "cd /app/env && uvicorn server.app:app --host 0.0.0.0 --port 8000"]

If you introduced extra dependencies in the Dockerfile, you should install them in the Dockerfile before removing temp files.

7. Build & Validate with the CLI#

From the environment directory:

cd envs/my_env
openenv build          # Builds Docker image (auto-detects context)
openenv validate --verbose

openenv build understands both standalone environments and in-repo ones. Useful flags:

--tag/-t: override the default openenv-<env_name> tag
--build-arg KEY=VALUE: pass multiple Docker build arguments
--dockerfile / --context: custom locations when experimenting
--no-cache: force fresh dependency installs

openenv validate checks for required files, ensures the Dockerfile/server entrypoints function, and lists supported deployment modes. The command exits non-zero if issues are found so you can wire it into CI.

9. Automate Builds (optional)#

To trigger Docker builds on every push to main, add your environment to the matrix in .github/workflows/docker-build.yml:

strategy:
  matrix:
    image:
      - name: echo-env
        dockerfile: envs/echo_env/server/Dockerfile
      - name: chat-env
        dockerfile: envs/chat_env/server/Dockerfile
      - name: coding-env
        dockerfile: envs/coding_env/server/Dockerfile
      - name: my-env  # Add your environment here
        dockerfile: envs/my_env/server/Dockerfile

Use Your Environment#

Here is a simple example of using your environment:

from envs.my_env import MyAction, MyEnv

# Create environment from Docker image
client = MyEnv.from_docker_image("my-env:latest")
# Or, connect to the remote space on Hugging Face
client = MyEnv.from_hub("my-org/my-env")
# Or, connect to the local server
client = MyEnv(base_url="http://localhost:8000")

# Use context manager for automatic cleanup (recommended)
with client:
    # Reset
    result = client.reset()
    print(result.observation.result)  # "Ready"

    # Execute actions
    result = client.step(MyAction(command="test", parameters={}))
    print(result.observation.result)
    print(result.observation.success)

    # Get state
    state = client.state()
    print(state.episode_id)
    print(state.step_count)

# Or manually manage the connection
try:
    client = MyEnv(base_url="http://localhost:8000")
    result = client.reset()
    result = client.step(MyAction(command="test", parameters={}))
finally:
    client.close()

Nice work! You’ve now built and used your own OpenEnv environment.#

Your next steps are to:

Try out the end-to-end tutorial

Packaging & Deploying#

Quick Reference Card#

CLI Quick Reference#

Overview#

Prerequisites#

Step-by-Step Guide#

1. Scaffold with `openenv init`#

2. Define Models#

3. Implement Environment Logic#

4. Create the FastAPI Server#

5. Implement the Client#

6. Configure Dependencies & Dockerfile#

7. Build & Validate with the CLI#

9. Automate Builds (optional)#

Use Your Environment#

Nice work! You’ve now built and used your own OpenEnv environment.#

Docs

Tutorials

Resources

Packaging & Deploying#

Quick Reference Card#

CLI Quick Reference#

Overview#

Prerequisites#

Step-by-Step Guide#

1. Scaffold with openenv init#

2. Define Models#

3. Implement Environment Logic#

4. Create the FastAPI Server#

5. Implement the Client#

6. Configure Dependencies & Dockerfile#

7. Build & Validate with the CLI#

8. Push & Share with openenv push#

9. Automate Builds (optional)#

Use Your Environment#

Nice work! You’ve now built and used your own OpenEnv environment.#

Docs

Tutorials

Resources

1. Scaffold with `openenv init`#

8. Push & Share with `openenv push`#