Skip to content

Commit

Permalink
fix: use effective stake (root + alpha), set 5000 for effective stake (…
Browse files Browse the repository at this point in the history 
…#142)

* fix: use effective stake (root + alpha), set 5000 for effective stake

* chore: update README
  • Loading branch information
@jarvis8x7b
jarvis8x7b authored Feb 25, 2025
1 parent 6e7aa3e commit 9d151fc
Show file tree
Hide file tree
Showing 11 changed files with 145 additions and 77 deletions.
3 changes: 0 additions & 3 deletions .env.example
View file
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,3 @@ VALIDATOR_TX_TIMEOUT=10

SIMULATION=false
FAST_MODE=false



57 changes: 30 additions & 27 deletions README-miner.md
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Mining

There are two options in setting up a miner for Dojo (centralised method and decentralised method)

Before starting, create a .env file by making a copy of .env.example
Expand All @@ -8,10 +9,11 @@ cp .env.example .env
```

### Option 1: Centralised Method

Complete the .env file by changing / uncommenting the required variables

| Variable | Description | Default Value | Remarks |
|---------------------|-------------------------------------------------------------------|---------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|
| ------------------- | ----------------------------------------------------------------- | ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| BITTENSOR_DIR | Bittensor directory | $HOME/.bittensor | |
| WALLET_COLDKEY | Bittensor coldkey name | - | |
| WALLET_HOTKEY | Bittensor hotkey name | - | |
Expand All @@ -21,7 +23,7 @@ Complete the .env file by changing / uncommenting the required variables
| DOJO_API_BASE_URL | Base URL for Dojo API | <https://dojo-api.tensorplex.ai> | Dojo Worker API URL |
| DOJO_API_KEY | Authentication key for Dojo API | - | Dojo API key (Generate it in the next step) |
| AXON_PORT | Port for Axon server | 8091 | |
| VALIDATOR_MIN_STAKE | Optional minimum stake requirement | 20000 | It is recommended to set value to 0 for testnet |
| VALIDATOR_MIN_STAKE | Validator Minimum Stake | 5000 | Minimum stake required from validators. It is recommended to set value to 0 for testnet |
| TASK_MAX_RESULTS | The number of workers that may submit responses for a single task | 4 | |

Run Dojo CLI to retrieve API Key and Subscription Key. See [Dojo CLI](#dojo-cli) for usage. Note down the API key, subscription key and append the API key to your .env file.
Expand Down Expand Up @@ -51,31 +53,32 @@ make miner

Complete the .env file by changing / uncommenting the required variables

| Variable | Description | Default Value | Remarks |
|-------------------------|-------------------------------|--------------------------------------------------------|-----------------------------------------|
| SAS_SUBSTRATE_URL | Substrate URL | Same as SUBTENSOR_ENDPOINT | Must match network configuration |
| SAS_EXPRESS_PORT | Sidecar Express server port | 8081 | Internal service port |
| SUBSTRATE_API_URL | Substrate API URL | sidecar:8081 | Internal service endpoint |
| NEXT_PUBLIC_BACKEND_URL | Backend URL for Dojo UI | <http://localhost:3000> | Must be accessible from UI |
| SERVER_PORT | Worker API server port | 8080 | Must not conflict with other services |
| RUNTIME_ENV | Runtime environment | aws | Options: local, development, production |
| CORS_ALLOWED_ORIGINS | Allowed CORS origins | <http://localhost*,http://worker-ui*,http://dojo-cli>* | Comma-separated list |
| TOKEN_EXPIRY | JWT token expiration in hours | 24 | Adjust if needed |
| JWT_SECRET | Secret key for JWT tokens | - | Use a strong random string |
| AWS_ACCESS_KEY_ID | AWS access key ID | - | Optional |
| AWS_SECRET_ACCESS_KEY | AWS secret access key | - | Optional |
| AWS_S3_BUCKET_NAME | S3 bucket name | - | Optional |
| S3_PUBLIC_URL | Public URL for S3 bucket | - | Optional |
| REDIS_HOST | Redis host | redis | Container name or IP |
| REDIS_PORT | Redis port | 6379 | Default Redis port |
| REDIS_USERNAME | Redis username | - | (Optional) For Redis ACL |
| REDIS_PASSWORD | Redis password | - | (Optional) For Redis authentication |
| DB_HOST | Database host address | postgres:5432 | Format: hostname:port |
| DB_NAME | Database name | db | Database Name |
| DB_USERNAME | Database username | - | Database Username |
| DB_PASSWORD | Database password | - | Database Password |
| VALIDATOR_MIN_STAKE | Validator Minimum Stake | 20000 | Minimum stake required from validators |
| ETHEREUM_NODE | Ethereum Node endpoint | <https://ethereum.publicnode.com> | Ethereum endpoint |
| Variable | Description | Default Value | Remarks |
| ----------------------- | ----------------------------- | ------------------------------------------------------- | --------------------------------------------------------------------------------------- |
| SAS_SUBSTRATE_URL | Substrate URL | Same as SUBTENSOR_ENDPOINT | Must match network configuration |
| SAS_EXPRESS_PORT | Sidecar Express server port | 8081 | Internal service port |
| SUBSTRATE_API_URL | Substrate API URL | sidecar:8081 | Internal service endpoint |
| NEXT_PUBLIC_BACKEND_URL | Backend URL for Dojo UI | <http://localhost:3000> | Must be accessible from UI |
| SERVER_PORT | Worker API server port | 8080 | Must not conflict with other services |
| RUNTIME_ENV | Runtime environment | aws | Options: local, development, production |
| CORS_ALLOWED_ORIGINS | Allowed CORS origins | <http://localhost*,http://worker-ui*,http://dojo-cli>\* | Comma-separated list |
| TOKEN_EXPIRY | JWT token expiration in hours | 24 | Adjust if needed |
| JWT_SECRET | Secret key for JWT tokens | - | Use a strong random string |
| AWS_ACCESS_KEY_ID | AWS access key ID | - | Optional |
| AWS_SECRET_ACCESS_KEY | AWS secret access key | - | Optional |
| AWS_S3_BUCKET_NAME | S3 bucket name | - | Optional |
| S3_PUBLIC_URL | Public URL for S3 bucket | - | Optional |
| REDIS_HOST | Redis host | redis | Container name or IP |
| REDIS_PORT | Redis port | 6379 | Default Redis port |
| REDIS_USERNAME | Redis username | - | (Optional) For Redis ACL |
| REDIS_PASSWORD | Redis password | - | (Optional) For Redis authentication |
| DB_HOST | Database host address | postgres:5432 | Format: hostname:port |
| DB_NAME | Database name | db | Database Name |
| DB_USERNAME | Database username | - | Database Username |
| DB_PASSWORD | Database password | - | Database Password |
| VALIDATOR_MIN_STAKE | Validator Minimum Stake | 5000 | Minimum stake required from validators. It is recommended to set value to 0 for testnet |
| ETHEREUM_NODE | Ethereum Node endpoint | <https://ethereum.publicnode.com> | Ethereum endpoint |

Start the dojo platform which Dojo CLI will interact with later.

```bash
Expand Down
12 changes: 4 additions & 8 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -43,14 +43,9 @@
- [Required Software](#required-software-1)
- [System Requirements](#system-requirements-1)
- [Getting Started](#getting-started)
- [Mining](#mining)
- [Option 1: Centralised Method](#option-1-centralised-method)
- [Option 2: Decentralised Method](#option-2-decentralised-method)
- [Setup Subscription Key for Labellers on UI to connect to Dojo Subnet for scoring](#setup-subscription-key-for-labellers-on-ui-to-connect-to-dojo-subnet-for-scoring)
- [Validating](#validating)
- [Data Collection](#data-collection)
- [For Miners](#for-miners)
- [For Validators](#for-validators)
- [Auto-updater](#auto-updater)
- [Dojo CLI](#dojo-cli)
- [For Dojo developers](#for-dojo-developers)
- [Dataset Extraction](#dataset-extraction)
- [License](#license)
Expand Down Expand Up @@ -209,7 +204,7 @@ docker plugin install grafana/loki-docker-driver:3.3.2-amd64 --alias loki --gran
```

4. Start local subtensor node (**optional**)
> The included subtensor service only expose 30333 (p2p) to the public, 9933 and 9944 are only accesssible internally in the docker network, feel free to change the configuration if required.
> The included subtensor service only expose 30333 (p2p) to the public, 9933 and 9944 are only accesssible internally in the docker network, feel free to change the configuration if required.
```bash
# Mainnet
Expand Down Expand Up @@ -262,6 +257,7 @@ btcli s register --wallet.name coldkey --wallet.hotkey hotkey --netuid 98 --subt
```

## [For Miners](README-miner.md)

## [For Validators](README-validator.md)

# Auto-updater
Expand Down
2 changes: 1 addition & 1 deletion commons/block_subscriber.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ async def monitor_subscription(
indicating the subscription has likely failed.
"""
while True:
await asyncio.sleep(BLOCK_TIME)
await asyncio.sleep(5 * BLOCK_TIME)
time_since_last = (datetime.now() - watchdog.last_block_time).total_seconds()

if not watchdog.check_health():
Expand Down
61 changes: 61 additions & 0 deletions commons/utils.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,71 @@
import plotext
import requests
import torch
from bittensor.core.metagraph import AsyncMetagraph
from bittensor.utils.btlogging import logging as logger
from Crypto.Hash import keccak
from tenacity import RetryError, Retrying, stop_after_attempt, wait_exponential_jitter

from commons.objects import ObjectManager

ROOT_WEIGHT = 0.18
ROOT_NETUID = 0


def get_effective_stake(hotkey: str, subtensor: bt.subtensor) -> float:
if isinstance(subtensor, bt.AsyncSubtensor):
raise NotImplementedError("Async subtensor not supported")

root_stake = 0
try:
root_metagraph = subtensor.metagraph(ROOT_NETUID)
root_stake = root_metagraph.S[root_metagraph.hotkeys.index(hotkey)].item()
except (ValueError, IndexError):
logger.trace(
f"Hotkey {hotkey} not found in root metagraph, defaulting to 0 root_stake"
)

alpha_stake = 0
try:
config = ObjectManager.get_config()
subnet_metagraph = subtensor.metagraph(netuid=config.netuid) # type:ignore
alpha_stake = subnet_metagraph.alpha_stake[
subnet_metagraph.hotkeys.index(hotkey)
]
except (ValueError, IndexError):
logger.trace(
f"Hotkey {hotkey} not found in subnet metagraph for netuid: {subnet_metagraph.netuid}, defaulting to 0 alpha_stake"
)

effective_stake = (root_stake * ROOT_WEIGHT) + alpha_stake

return effective_stake


async def aget_effective_stake(
hotkey: str, root_metagraph: AsyncMetagraph, subnet_metagraph: AsyncMetagraph
) -> float:
root_stake = 0
try:
root_stake = root_metagraph.S[root_metagraph.hotkeys.index(hotkey)].item()
except (ValueError, IndexError):
logger.trace(
f"Hotkey {hotkey} not found in root metagraph, defaulting to 0 root_stake"
)

alpha_stake = 0
try:
idx = subnet_metagraph.hotkeys.index(hotkey)
alpha_stake = subnet_metagraph.alpha_stake[idx]
except (ValueError, IndexError):
logger.trace(
f"Hotkey {hotkey} not found in subnet metagraph for netuid: {subnet_metagraph.netuid}, defaulting to 0 alpha_stake"
)

effective_stake = (root_stake * ROOT_WEIGHT) + alpha_stake

return effective_stake


def _terminal_plot(
title: str, y: np.ndarray, x: np.ndarray | None = None, sort: bool = False
Expand Down
8 changes: 4 additions & 4 deletions docker-compose.miner.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ networks:
name: subtensor

services:
# watchtower:
# extends:
# file: docker-compose.shared.yaml
# service: watchtower
# watchtower:
# extends:
# file: docker-compose.shared.yaml
# service: watchtower

dojo-cli:
container_name: dojo-cli
Expand Down
4 changes: 2 additions & 2 deletions dojo/__init__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ def get_spec_version():
)


VALIDATOR_MIN_STAKE = int(os.getenv("VALIDATOR_MIN_STAKE", "20000"))
VALIDATOR_MIN_STAKE = int(os.getenv("VALIDATOR_MIN_STAKE", "5000"))
TASK_DEADLINE = 6 * 60 * 60

# Define the time intervals for various tasks.
Expand All @@ -64,7 +64,7 @@ def get_spec_version():

if get_config().fast_mode:
print("Running in fast mode for testing purposes...")
VALIDATOR_MIN_STAKE = int(os.getenv("VALIDATOR_MIN_STAKE", "20000"))
VALIDATOR_MIN_STAKE = int(os.getenv("VALIDATOR_MIN_STAKE", "5000"))
TASK_DEADLINE = 180
VALIDATOR_RUN = 60
VALIDATOR_HEARTBEAT = 15
Expand Down
Loading

0 comments on commit 9d151fc

Please sign in to comment.