MIBs processed with mibdump cannot be imported with load_module

[desultory]

Expected behavior

with the mib from: https://download.mikrotik.com/routeros/7.17.2/mikrotik.mib

Using:

mibdump --destination-template pysnmp/mib-instrumentation/managed-objects-instances.j2 --destination-directory /usr/lib/python3.12/site-packages/pysnmp/smi/mibs .snmp/mibs/mikrotik.mib

should allow me to use load_module("MIKROTIK-MIB")

Actual behavior

pysnmp.smi.error.MibLoadError: MIB module '/usr/lib/python3.12/site-packages/pysnmp/smi/mibs/MIKROTIK-MIB.py' load error: ['Traceback (most recent call last):\n', '  File "/usr/lib/python3.12/site-packages/pysnmp/
smi/builder.py", line 364, in load_module\n    runpy.run_path(\n', '  File "<frozen runpy>", line 287, in run_path\n', '  File "<frozen runpy>", line 98, in _run_module_code\n', '  File "<frozen runpy>", line 88, 
in _run_code\n', '  File "/usr/lib/python3.12/site-packages/pysnmp/smi/mibs/MIKROTIK-MIB.py", line 419, in <module>\n    mtxrWlStatTxRate) = mibBuilder.importSymbols(\n                        ^^^^^^^^^^^^^^^^^^^^^
^^^^\n', '  File "/usr/lib/python3.12/site-packages/pysnmp/smi/builder.py", line 462, in import_symbols\n    self.load_modules(modName, **userCtx)\n', '  File "/usr/lib/python3.12/site-packages/pysnmp/smi/builder.
py", line 410, in load_modules\n    self.load_module(modName, **userCtx)\n', '  File "/usr/lib/python3.12/site-packages/pysnmp/smi/builder.py", line 386, in load_module\n    raise error.MibNotFoundError(\n', 'pysn
mp.smi.error.MibNotFoundError: MIB file "mtxrAltitude.py[co]" not found in search path (DirMibSource(\'/usr/lib/python3.12/site-packages/pysnmp/smi/mibs\'), DirMibSource(\'/usr/lib/python3.12/site-packages/pysnmp/
smi/mibs/instances\'), DirMibSource(\'pysnmp_mibs\'))\n'] caused by <class 'pysnmp.smi.error.MibNotFoundError'>: MIB file "mtxrAltitude.py[co]" not found in search path (DirMibSource('/usr/lib/python3.12/site-pack
ages/pysnmp/smi/mibs'), DirMibSource('/usr/lib/python3.12/site-packages/pysnmp/smi/mibs/instances'), DirMibSource('pysnmp_mibs'))

Detailed steps

wget https://download.mikrotik.com/routeros/7.17.2/mikrotik.mib
mibdump --destination-template pysnmp/mib-instrumentation/managed-objects-instances.j2 --destination-directory /usr/lib/python3.12/site-packages/pysnmp/smi/mibs mikrotik.mib

engine = SnmpEngine()
mib_builder = engine.get_mib_builder()
mib_builder.load_module("MIKROTIK-MIB")

Python package information

7.1.16

Operating system information

gentoo linux

Python information

Python 3.12.8 (main, Jan 18 2025, 11:16:35) [GCC 14.2.1 20241221] on linux

(Optional) Contents of your test script

from enum import Enum
from pathlib import Path

from pysnmp.hlapi.v3arch.asyncio import (
    USM_AUTH_HMAC96_MD5,
    USM_AUTH_HMAC96_SHA,
    USM_AUTH_HMAC192_SHA256,
    USM_AUTH_HMAC384_SHA512,
    USM_PRIV_CBC168_3DES,
    USM_PRIV_CFB128_AES,
    USM_PRIV_CFB192_AES,
    USM_PRIV_CFB256_AES,
    CommunityData,
    ContextData,
    ObjectIdentity,
    ObjectType,
    SnmpEngine,
    UdpTransportTarget,
    UsmUserData,
    get_cmd,
    walk_cmd,
)
from pysnmp.smi import view
from pysnmp.smi.builder import DirMibSource
from zenlib.logging import loggify

DEFAULT_ARGS = [
    {"flags": ["target"], "help": "Target IP address or hostname"},
    {"flags": ["oid"], "help": "OID to query"},
    {"flags": ["-p", "--port"], "help": "SNMP port", "default": 161},
    {"flags": ["-c", "--community"], "help": "SNMP community string"},
    {"flags": ["-u", "--user"], "help": "SNMPv3 username"},
    {"flags": ["-X", "--encryption-key"], "help": "SNMPv3 encryption key"},
    {"flags": ["-x", "--encryption-protocol"], "help": "SNMPv3 encryption protocol"},
    {"flags": ["-A", "--auth-key"], "help": "SNMPv3 auth key"},
    {"flags": ["-a", "--auth-protocol"], "help": "SNMPv3 auth protocol"},
]


class AuthTypes(Enum):
    MD5 = USM_AUTH_HMAC96_MD5
    SHA = USM_AUTH_HMAC96_SHA
    SHA256 = USM_AUTH_HMAC192_SHA256
    SHA512 = USM_AUTH_HMAC384_SHA512


class EncryptionTypes(Enum):
    DES = USM_PRIV_CBC168_3DES
    AES128 = USM_PRIV_CFB128_AES
    AES192 = USM_PRIV_CFB192_AES
    AES256 = USM_PRIV_CFB256_AES


class SNMPError(Exception):
    pass


@loggify
class ZenSNMP:
    def __init__(
        self,
        target,
        port=161,
        community="public",
        user=None,
        encryption_key=None,
        encryption_protocol=None,
        auth_key=None,
        auth_protocol=None,
        *args,
        **kwargs,
    ):
        self.target = target
        self.port = port
        self.encryption_key = encryption_key
        self.encryption_protocol = encryption_protocol or "AES128"
        self.auth_key = auth_key
        self.auth_protocol = auth_protocol or "SHA"
        self.user = user
        self.community_name = community

    @property
    def v3(self):
        if self.user is not None:
            if self.encryption_key is None:
                raise ValueError("Encryption key required for SNMPv3")
            if self.auth_key is None:
                raise ValueError("Auth key required for SNMPv3")
            return True
        return False

    @property
    def auth(self):
        if self.v3:
            return UsmUserData(
                self.user,
                self.auth_key,
                self.encryption_key,
                authProtocol=self.auth_protocol,
                privProtocol=self.encryption_protocol,
            )
        return CommunityData(self.community_name)

    @property
    def encryption_protocol(self):
        return EncryptionTypes[self._encryption_protocol.upper()].value

    @encryption_protocol.setter
    def encryption_protocol(self, value):
        if value.upper() not in EncryptionTypes.__members__:
            raise ValueError(f"Invalid encryption protocol: {value}")
        self._encryption_protocol = value

    @property
    def auth_protocol(self):
        return AuthTypes[self._auth_protocol.upper()].value

    @auth_protocol.setter
    def auth_protocol(self, value):
        if value.upper() not in AuthTypes.__members__:
            raise ValueError(f"Invalid auth protocol: {value}")
        self._auth_protocol = value

    async def init_snmp(self):
        self.transport = await UdpTransportTarget.create((self.target, self.port))
        self.context = ContextData()
        self.engine = SnmpEngine()
        # Load the MIKROTIK-MIB mib
        mib_builder = self.engine.get_mib_builder()
        # mib_sources = mib_builder.get_mib_sources() + (DirMibSource(Path("~/.pysnmp/mibs").expanduser()),)
        # mib_builder.set_mib_sources(*mib_sources)
        mib_builder.load_module("MIKROTIK-MIB")
        print("asdf")

    def get_oid_name(self, oid):
        builder = self.engine.get_mib_builder()
        oid_base, sym_name, oid_suffix = view.MibViewController(builder).getNodeName(oid)
        if str(oid_suffix) != "0":
            return sym_name[-1] + f".{oid_suffix}"
        return sym_name[-1]

    def get_object_bind_strings(self, binds):
        for bind in binds:
            name = self.get_oid_name(bind[0])
            val = bind[1].prettyPrint()
            yield f"[{bind[0]}] {name}: {val} ({type(bind[1]).__name__})"

    def handle_object(self, obj):
        error_indication, error_status, error_index, var_binds = obj
        if error_indication:
            self.logger.error(f"Error:\n{error_indication}")
            self.logger.error(f"Error status:\n{error_status}")
            self.logger.error(f"Error index:\n{error_index}")
            raise SNMPError(error_indication)
        for string in self.get_object_bind_strings(var_binds):
            self.logger.info(string)

    async def get_oid(self, oid):
        self.logger.info(f"Getting OID: {oid}")
        obj = ObjectType(ObjectIdentity(oid))
        result = await get_cmd(self.engine, self.auth, self.transport, self.context, obj)
        self.handle_object(result)
        return result

    async def walk_oid(self, oid):
        self.logger.info(f"Walking OID: {oid}")
        obj = ObjectType(ObjectIdentity(oid))
        objects = []
        async for item in walk_cmd(self.engine, self.auth, self.transport, self.context, obj):
            try:
                self.handle_object(item)
            except SNMPError as e:
                self.logger.error(f"[{oid}] Error walking object: {e}")
                continue
            objects.append(item)
        self.logger.debug(f"[{oid}] Walk result:\n{objects}")
        return objects

Relevant log output

Loading MIB: MIKROTIK-MIB from: DirMibSource('/usr/lib/python3.12/site-packages/pysnmp/smi/mibs')
Loading module symbols: SNMPv2-SMI
Loading module symbols: mtxrAltitude
Loading MIB: mtxrAltitude from: DirMibSource('/usr/lib/python3.12/site-packages/pysnmp/smi/mibs')
loadModule: read mtxrAltitude from DirMibSource('/usr/lib/python3.12/site-packages/pysnmp/smi/mibs') failed: [Errno 2] No suitable module found: 'mtxrAltitude'
Loading MIB: mtxrAltitude from: DirMibSource('/usr/lib/python3.12/site-packages/pysnmp/smi/mibs/instances')
loadModule: read mtxrAltitude from DirMibSource('/usr/lib/python3.12/site-packages/pysnmp/smi/mibs/instances') failed: [Errno 2] No suitable module found: 'mtxrAltitude'
Loading MIB: mtxrAltitude from: DirMibSource('pysnmp_mibs')
loadModule: read mtxrAltitude from DirMibSource('pysnmp_mibs') failed: [Errno 2] No suitable module found: 'mtxrAltitude'
{'ASN1': '/usr/lib/python3.12/site-packages/pysnmp/smi/mibs/ASN1.py', 'ASN1-ENUMERATION': '/usr/lib/python3.12/site-packages/pysnmp/smi/mibs/ASN1-ENUMERATION.py', 'ASN1-REFINEMENT': '/usr/lib/python3.12/site-packa
ges/pysnmp/smi/mibs/ASN1-REFINEMENT.py', 'SNMPv2-SMI': '/usr/lib/python3.12/site-packages/pysnmp/smi/mibs/SNMPv2-SMI.py', 'SNMPv2-CONF': '/usr/lib/python3.12/site-packages/pysnmp/smi/mibs/SNMPv2-CONF.py', 'SNMPv2-
TC': '/usr/lib/python3.12/site-packages/pysnmp/smi/mibs/SNMPv2-TC.py', 'SNMPv2-MIB': '/usr/lib/python3.12/site-packages/pysnmp/smi/mibs/SNMPv2-MIB.py', 'SNMP-MPD-MIB': '/usr/lib/python3.12/site-packages/pysnmp/smi
/mibs/SNMP-MPD-MIB.py', 'SNMP-FRAMEWORK-MIB': '/usr/lib/python3.12/site-packages/pysnmp/smi/mibs/SNMP-FRAMEWORK-MIB.py', 'SNMP-TARGET-MIB': '/usr/lib/python3.12/site-packages/pysnmp/smi/mibs/SNMP-TARGET-MIB.py', '
SNMP-COMMUNITY-MIB': '/usr/lib/python3.12/site-packages/pysnmp/smi/mibs/SNMP-COMMUNITY-MIB.py', 'SNMP-USER-BASED-SM-MIB': '/usr/lib/python3.12/site-packages/pysnmp/smi/mibs/SNMP-USER-BASED-SM-MIB.py', '__SNMP-FRAM
EWORK-MIB': '/usr/lib/python3.12/site-packages/pysnmp/smi/mibs/instances/__SNMP-FRAMEWORK-MIB.py'}
Traceback (most recent call last):
  File "<input>", line 1, in <module>
    mib_builder.load_module("MIKROTIK-MIB")
  File "/usr/lib/python3.12/site-packages/pysnmp/smi/builder.py", line 372, in load_module
    raise error.MibLoadError(
pysnmp.smi.error.MibLoadError: MIB module '/usr/lib/python3.12/site-packages/pysnmp/smi/mibs/MIKROTIK-MIB.py' load error: ['Traceback (most recent call last):\n', '  File "/usr/lib/python3.12/site-packages/pysnmp/
smi/builder.py", line 364, in load_module\n    runpy.run_path(\n', '  File "<frozen runpy>", line 287, in run_path\n', '  File "<frozen runpy>", line 98, in _run_module_code\n', '  File "<frozen runpy>", line 88, 
in _run_code\n', '  File "/usr/lib/python3.12/site-packages/pysnmp/smi/mibs/MIKROTIK-MIB.py", line 419, in <module>\n    mtxrWlStatTxRate) = mibBuilder.importSymbols(\n                        ^^^^^^^^^^^^^^^^^^^^^
^^^^\n', '  File "/usr/lib/python3.12/site-packages/pysnmp/smi/builder.py", line 462, in import_symbols\n    self.load_modules(modName, **userCtx)\n', '  File "/usr/lib/python3.12/site-packages/pysnmp/smi/builder.
py", line 410, in load_modules\n    self.load_module(modName, **userCtx)\n', '  File "/usr/lib/python3.12/site-packages/pysnmp/smi/builder.py", line 386, in load_module\n    raise error.MibNotFoundError(\n', 'pysn
mp.smi.error.MibNotFoundError: MIB file "mtxrAltitude.py[co]" not found in search path (DirMibSource(\'/usr/lib/python3.12/site-packages/pysnmp/smi/mibs\'), DirMibSource(\'/usr/lib/python3.12/site-packages/pysnmp/
smi/mibs/instances\'), DirMibSource(\'pysnmp_mibs\'))\n'] caused by <class 'pysnmp.smi.error.MibNotFoundError'>: MIB file "mtxrAltitude.py[co]" not found in search path (DirMibSource('/usr/lib/python3.12/site-pack
ages/pysnmp/smi/mibs'), DirMibSource('/usr/lib/python3.12/site-packages/pysnmp/smi/mibs/instances'), DirMibSource('pysnmp_mibs'))

[lextudio-support]

New issues are marked as low priority by default. Becoming our commercial customers, and then your reports are handled with higher priority after triage.

[lextm]

I think this question stems from how you interpreted the documentation and samples. Here’s a structured way to approach the problem:

1. Compiling MIB Documents

You need to call mibdump with the correct parameters to actually compile the MIB documents into .py files. Based on what you've shared, it seems the wrong options were used, preventing successful compilation.

The output of mibdump can sometimes be confusing, but experimenting with different options will help clarify how it works.

2. Loading Compiled MIBs in PySNMP

Once you confirm that the desired .py files have been generated in the correct directory by mibdump, you can proceed with PySNMP to load and use them.

We understand that MIB compilation and PySNMP usage involve a learning curve, and while we aim to provide documentation and examples, we have limited resources to offer in-depth guidance for every scenario. However, you can find extensive knowledge accumulated over the years from online resources and search engines.

If you require dedicated support or expert assistance, we offer consulting services to help with more complex use cases. You can find more details here:
https://docs.lextudio.com/snmp/support

[desultory]

I think this question stems from how you interpreted the documentation and samples. Here’s a structured way to approach the problem:

1. Compiling MIB Documents

You need to call mibdump with the correct parameters to actually compile the MIB documents into .py files. Based on what you've shared, it seems the wrong options were used, preventing successful compilation.

The output of mibdump can sometimes be confusing, but experimenting with different options will help clarify how it works.

2. Loading Compiled MIBs in PySNMP

Once you confirm that the desired .py files have been generated in the correct directory by mibdump, you can proceed with PySNMP to load and use them.

We understand that MIB compilation and PySNMP usage involve a learning curve, and while we aim to provide documentation and examples, we have limited resources to offer in-depth guidance for every scenario. However, you can find extensive knowledge accumulated over the years from online resources and search engines.

If you require dedicated support or expert assistance, we offer consulting services to help with more complex use cases. You can find more details here: https://docs.lextudio.com/snmp/support

it successfully "compiled" a .py file, which pysnmp failed to import. I'm not sure how to fix this, and I've read those pages.

to be completely honest, I'm not sure it's the best method to offer paid assistance when you instantly mark an unresolved ticket as completed. Maybe you can't help here, but surely in this community project someone else may stop by and help?

paid support is the type of thing i'd consider if I was able to get get something working, not an option I'd consider before I can even get a PoC with this library.

[dcvmoole]

Hello!

If I may: are you sure that you want to use the --destination-template mibdump option? The result of that option is that the produced file is not a regular compiled MIB file, and cannot be expected to be loaded as one. Such a template-generated file will need to be renamed as it tries to import from the regular compiled MIB file, which in this case you do not have. As far as I understand, these kind of template-produced files are useful only for (hand-extended) use in the context of an SNMP agent, and it looks like your code is a client program.

Even in that light, I do not understand why you are getting the error you see, because the object name mtxrAltitude should never be loaded as a file. I believe the expected result of your current approach is that the compiled file will fail to import from itself.

[desultory]

Hello!

If I may: are you sure that you want to use the --destination-template mibdump option? The result of that option is that the produced file is not a regular compiled MIB file, and cannot be expected to be loaded as one. Such a template-generated file will need to be renamed as it tries to import from the regular compiled MIB file, which in this case you do not have. As far as I understand, these kind of template-produced files are useful only for (hand-extended) use in the context of an SNMP agent, and it looks like your code is a client program.

Yes, I think I mixed those parts up.

Even in that light, I do not understand why you are getting the error you see, because the object name mtxrAltitude should never be loaded as a file. I believe the expected result of your current approach is that the compiled file will fail to import from itself.

I'm not sure exactly what is going on, maybe you can replicate with the steps I used before.

Trying your suggestion:

sudo mibdump --destination-directory /usr/lib/python3.12/site-packages/pysnmp/smi/mibs .snmp/mibs/mikrotik.mib
Source MIB repositories: file:///home/desu/.snmp/mibs, file:///usr/share/snmp/mibs, https://mibs.pysnmp.com/asn1/@mib@
Borrow missing/failed MIBs from: https://mibs.pysnmp.com:443/mibs/notexts/@mib@
Existing/compiled MIB locations: pysnmp.smi.mibs, pysnmp_mibs
Compiled MIBs destination directory: /usr/lib/python3.12/site-packages/pysnmp/smi/mibs
MIBs excluded from code generation: INET-ADDRESS-MIB, PYSNMP-USM-MIB, RFC-1212, RFC-1215, RFC1065-SMI, RFC1155-SMI, RFC1158-MIB, RFC1213-MIB, SNMP-FRAMEWORK-MIB, SNMP-TARGET-MIB, SNMPv2-CONF, SNMPv2-SMI, SNMPv2-TC, SNMPv2-TM, TRANSPORT-ADDRESS-MIB
MIBs to compile: mikrotik
Destination format: pysnmp
Custom destination template: None
Parser grammar cache directory: not used
Also compile all relevant MIBs: yes
Rebuild MIBs regardless of age: no
Dry run mode: no
Create/update MIBs: yes
Byte-compile Python modules: yes (optimization level no)
Ignore compilation errors: no
Generate OID->MIB index: no
Generate texts in MIBs: no
Keep original texts layout: no
Try various file names while searching for MIB module: yes
Created/updated MIBs: MIKROTIK-MIB (mikrotik)
Pre-compiled MIBs borrowed: 
Up to date MIBs: INET-ADDRESS-MIB, SNMPv2-CONF, SNMPv2-SMI, SNMPv2-TC
Missing source MIBs: 
Ignored MIBs: 
Failed MIBs: 

I think it may be working now, thank you very much, I think the docs could really use a super simple "compile mib into usable format" type of thing, it seems possible with a few steps but I was bouncing between pages for hours because a lot of the info seems dedicated to creating custom MIB definitions, etc.