Add Nle env to OpenEnv #161

init27 · 2025-11-07T03:54:50Z

Add NetHack Learning Environment (NLE) Integration

Summary

Adds a complete integration of the NetHack Learning Environment (NLE) as an OpenEnv environment.

What's Added

Core Implementation (8 files, ~600 LOC)

Models (models.py):

NLEAction - Discrete action space (113 actions: movement, commands, magic, inventory)
NLEObservation - Rich observation space with 14+ types (~140KB JSON per step)
NLEState - Episode state tracking (step count, game status, character info)

Client (client.py):

NLEEnv - HTTP client implementing HTTPEnvClient[NLEAction, NLEObservation]
Handles both old gym (0.25) and new gym (0.26+) API compatibility
Supports from_docker_image() for automatic container deployment

Server (server/):

NLEEnvironment - Environment wrapper implementing OpenEnv Environment ABC
Wraps NLE's gym interface with reset(), step(), state property
app.py - FastAPI application using create_app()
Dockerfile - Python 3.11 with NLE compiled from source

Environment Characteristics

Action Space:

Type: Discrete (113 actions, 0-112)
Categories: Movement (8), Diagonal (8), Commands (97)
Examples: Move north (0), Eat (37), Search (50), Inventory (104)

Observation Space (~140KB/step):

glyphs (21×79): Symbolic dungeon map
blstats (26): HP, XP, gold, dungeon level, etc.
message (256): Game message buffer
chars, colors, specials (21×79 each): Visual display
inv_glyphs, inv_strs, inv_letters, inv_oclasses: Inventory
tty_chars, tty_colors, tty_cursor: Terminal display

Reward:

Default: Score delta (current_score - previous_score)
Range: Unbounded (positive for progress)

Episode Termination:

Death (player dies)
Ascension (player wins - very rare!)
Aborted (max steps reached, default 5000)

Design Decisions & Assumptions

1. Assumption: Beefy Compute Available

Decision: Use simple JSON serialization, include all observations by default

2. Gym API Compatibility Layer

Decision: Support both gym 0.25 and 0.26+ APIs

3. Python 3.11 Requirement

4. Single Task Implementation

Decision: Implement only NetHackScore task (maximize score)

What Was Skipped / Deferred

Phase 2 Features (Low Priority)

These can be added later if needed:

Task Variants
- NetHackStaircase (reach stairs)
- NetHackOracle (find oracle)
- NetHackGold (collect gold)
- NetHackEat (maximize hunger)
- NetHackScout (maximize exploration)
Seeding API
- NLE supports deterministic seeding for reproducibility
- env.seed(core=42, disp=123, reseed=False)
TTYrec Recording/Replay
- NLE can save game recordings (ttyrec files)
- Currently disabled by default (save_ttyrec_every=0)
Wizard Mode
- Debug mode for testing (spawn items, teleport, etc.)
Performance Optimizations
- Binary serialization (msgpack)
- Compression (gzip)
- Selective observations

References

NLE GitHub: https://github.com/facebookresearch/nle
NLE Paper (NeurIPS 2020): https://arxiv.org/abs/2006.13760
NetHack Wiki: https://nethackwiki.com
OpenEnv RFC 002: rfcs/002-env-spec.md

github-actions · 2025-11-07T04:00:56Z

✅ Deployment succeeded for nle_env

Space repo: https://huggingface.co/spaces/openenv-testing/nle_env-pr-161
Live URL: https://openenv-testing-nle-env-pr-161.hf.space

Nice work! Wait for a code review and we're ready to go.

You can iterate locally or validate fixes by running scripts/deploy_to_hf.sh --env "nle_env".

Copilot

Pull Request Overview

This PR adds a NetHack Learning Environment (NLE) integration to OpenEnv, providing HTTP-based access to the challenging roguelike game NetHack 3.6.6 for reinforcement learning research.

Key changes:

Implements NLE environment wrapper with full observation space support (14+ observation types)
Provides HTTP server/client architecture for remote environment access
Includes Docker containerization with NLE compilation from source

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/envs/nle_env/server/nle_environment.py	Core environment implementation wrapping NLE's gym interface with OpenEnv's Environment ABC
src/envs/nle_env/server/app.py	FastAPI application setup with environment variable configuration
src/envs/nle_env/server/init.py	Server module initialization
src/envs/nle_env/server/Dockerfile	Docker build configuration with NLE compilation and dependencies
src/envs/nle_env/models.py	Data models for actions, observations, and state
src/envs/nle_env/client.py	HTTP client for interacting with NLE server
src/envs/nle_env/init.py	Package initialization and exports
src/envs/nle_env/README.md	Comprehensive documentation for the NLE environment
examples/nle_random_agent.py	Example script demonstrating random agent gameplay

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-07T18:23:09Z

src/envs/nle_env/models.py

+@dataclass
+class NLEAction(Action):


The base class Action uses @dataclass(kw_only=True), so child classes should also explicitly use @dataclass(kw_only=True) for consistency with other environments in the codebase (e.g., echo_env, browsergym_env, finrl_env). Change to @dataclass(kw_only=True).

Copilot · 2025-11-07T18:23:09Z

src/envs/nle_env/models.py

+    action_id: int  # Index into nethack.USEFUL_ACTIONS (0-112)
+
+
+@dataclass


The base class Observation uses @dataclass(kw_only=True), so child classes should also explicitly use @dataclass(kw_only=True) for consistency with other environments in the codebase (e.g., echo_env, browsergym_env, finrl_env). Change to @dataclass(kw_only=True).

Suggested change

@dataclass

@dataclass(kw_only=True)

Copilot · 2025-11-07T18:23:10Z

src/envs/nle_env/README.md

+
+```bash
+# Build from repository root (not from server directory)
+cd /Users/sanyambhutani/GH/OpenEnv


This hardcoded absolute path should be replaced with a generic placeholder path. Consider using a relative path or a generic path like cd /path/to/OpenEnv or just refer to the 'repository root'.

Suggested change

cd /Users/sanyambhutani/GH/OpenEnv

cd /path/to/OpenEnv # Replace with the path to your repository root

src/envs/nle_env/models.py

Copilot · 2025-11-07T18:23:11Z

src/envs/nle_env/models.py

+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any, Dict, List, Optional


Import of 'Any' is not used.
Import of 'Dict' is not used.

Suggested change

from typing import Any, Dict, List, Optional

from typing import List, Optional

Darktex

Excellent implementation with strong documentation. 3 critical issues must be fixed before merge.

See inline comments for specific fixes needed.

Darktex · 2025-11-07T18:44:56Z

src/envs/nle_env/models.py

+        - action_id=50: Search (s)
+    """
+
+    action_id: int  # Index into nethack.USEFUL_ACTIONS (0-112)


🔴 CRITICAL: Missing Action Validation

No validation that action_id is within valid range (0-112). Invalid actions could cause runtime errors.

Suggested Fix:

@dataclass class NLEAction(Action): action_id: int # Index into nethack.USEFUL_ACTIONS (0-112) def __post_init__(self): if not 0 <= self.action_id <= 112: raise ValueError(f"action_id must be in range 0-112, got {self.action_id}")

Darktex · 2025-11-07T18:44:56Z

src/envs/nle_env/README.md

+
+```bash
+# Build from repository root (not from server directory)
+cd /Users/sanyambhutani/GH/OpenEnv


🔴 CRITICAL: Hardcoded User Path

This is a user-specific absolute path that won't work for others.

Fix:

# Build from repository root cd /path/to/OpenEnv # or cd $(git rev-parse --show-toplevel)

Darktex · 2025-11-07T18:44:56Z

src/envs/nle_env/client.py

+over HTTP.
+"""
+
+from typing import Dict


🔴 CRITICAL: Incomplete Type Annotations

Dict is used without full type specification throughout this file (lines 14, 57, 71, 129).

Fix:

from typing import Any, Dict

Then update all method signatures:

_step_payload() -> Dict[str, Any]

_parse_result(payload: Dict[str, Any]) -> ...

_parse_state(payload: Dict[str, Any]) -> ...

Darktex · 2025-11-07T18:44:56Z

src/envs/nle_env/server/nle_environment.py

+        """
+        super().__init__(transform=transform)
+
+        if NLE is None:


🟡 IMPORTANT: Improve Error Message

Error message should clarify Docker vs local development context.

Suggested:

if NLE is None: raise ImportError( "NLE is not installed. This should not happen in Docker deployments.\n" "For local development, install with: pip install nle\n" "For Docker builds, verify the build completed successfully." )

Darktex · 2025-11-07T18:44:56Z

src/envs/nle_env/server/__init__.py

+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""Server module for NLE environment."""


🟡 IMPORTANT: Missing __all__ Export

For consistency with the main package, consider adding:

from .nle_environment import NLEEnvironment __all__ = ["NLEEnvironment"]

Darktex · 2025-11-07T18:44:56Z

src/envs/nle_env/server/nle_environment.py

+        """
+        Convert NLE gym observation to NLEObservation.
+
+        With beefy compute, we just convert numpy arrays to lists.


🟢 MINOR: Informal Comment

"beefy compute" is a bit informal for production code. Consider:

# Convert numpy arrays to nested lists for JSON serialization. # This approach prioritizes simplicity over optimization, which is # acceptable given the compute resources available.

Darktex · 2025-11-07T18:44:57Z

src/envs/nle_env/server/app.py

+if __name__ == "__main__":
+    import uvicorn
+
+    # NLE must run single-threaded (workers=1) due to C extension


🟢 MINOR: Clarify Threading Comment

Explain why single-threading is needed:

# NLE must run single-threaded (workers=1) because the NetHack C extension # uses global state and is not thread-safe

Darktex · 2025-11-07T18:44:57Z

examples/nle_random_agent.py

@@ -0,0 +1,126 @@
+#!/usr/bin/env python3


✅ POSITIVE: Excellent Example

This example is clear, well-documented, and demonstrates proper usage. Great work!

Darktex · 2025-11-07T18:44:57Z

src/envs/nle_env/README.md

@@ -0,0 +1,316 @@
+# NetHack Learning Environment (NLE) for OpenEnv


✅ POSITIVE: Outstanding Documentation

This README is comprehensive, well-structured, and includes multiple examples. Excellent work on documentation!

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

github-actions · 2025-11-07T21:23:17Z

✅ Deployment succeeded for nle_env

Space repo: https://huggingface.co/spaces/openenv-testing/nle_env-pr-161
Live URL: https://openenv-testing-nle-env-pr-161.hf.space

Nice work! Wait for a code review and we're ready to go.

You can iterate locally or validate fixes by running scripts/deploy_to_hf.sh --env "nle_env".

init27 added 14 commits November 6, 2025 19:43

add example

0f234cf

Create __init__.py

1f5437a

Create client.py

9fb20dd

Create models.py

1f1aa30

Create README.md

41c5203

Create __init__.py

73b5687

Create app.py

ea73ebe

Create Dockerfile

92e482e

Create nle_environment.py

60fac83

Create test_integration.py

0867881

Create validate_structure.py

8e89fec

Delete test_integration.py

7e519c7

Delete validate_structure.py

eb7a0eb

mv fikes

6dc12cc

meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Nov 7, 2025

init27 requested review from Darktex and zkwentz November 7, 2025 03:55

Darktex requested a review from Copilot November 7, 2025 18:19

Copilot AI reviewed Nov 7, 2025

View reviewed changes

Darktex requested changes Nov 7, 2025

View reviewed changes

Update src/envs/nle_env/models.py

0614916

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

		action_id: int # Index into nethack.USEFUL_ACTIONS (0-112)


		@dataclass

	cd /Users/sanyambhutani/GH/OpenEnv
	cd /path/to/OpenEnv # Replace with the path to your repository root

	from typing import Any, Dict, List, Optional
	from typing import List, Optional

		@@ -0,0 +1,316 @@
		# NetHack Learning Environment (NLE) for OpenEnv

Add Nle env to OpenEnv #161

Are you sure you want to change the base?

Add Nle env to OpenEnv #161

Uh oh!

Conversation

init27 commented Nov 7, 2025

Add NetHack Learning Environment (NLE) Integration

Summary

What's Added

Core Implementation (8 files, ~600 LOC)

Environment Characteristics

Design Decisions & Assumptions

1. Assumption: Beefy Compute Available

2. Gym API Compatibility Layer

3. Python 3.11 Requirement

4. Single Task Implementation

What Was Skipped / Deferred

Phase 2 Features (Low Priority)

References

Uh oh!

github-actions bot commented Nov 7, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

Darktex left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Nov 7, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants