Klamp't

Klamp't (Kris' Locomotion and Manipulation Planning Toolbox) is an open-source, cross-platform software package for robot modeling, simulating, planning, optimization, and visualization. It aims to provide an accessible, wide range of programming tools for learning robotics, analyzing robots, developing algorithms, and prototyping intelligent behaviors. It has particular strengths in robot manipulation and locomotion.

Historically, it began development at Indiana University since 2009 primarily as a research platform. Since then, it has been adopted in education and research labs around the world.

More information can be found on the Klamp't website (http://klampt.org)

• Klamp't
  • Features
  • Installation
  • Documentation
  • Reporting bugs and getting help
  • Version history
  • Who uses Klamp't?
  • Comparison to related packages
  • Contributors

Features

• Unified C++ and Python package for robot modeling, kinematics, dynamics, control, motion planning, simulation, and visualization.
• Interoperable with Robot Operating System (ROS 1) and Open Motion Planning Library (OMPL).
• Stable file formats and tooling to save, load, and visualize robots (URDF), meshes, configurations, trajectories, poses, and more.
• Built-in conversions to and from Numpy, JSON, ROS 1, Open3D, trimesh, PyTorch, and Sympy objects (in the Python API).
• Many geometry types implemented, including meshes, point clouds, signed distance functions, occupancy grids, geometric primitives, and convex polytopes. Collision, distance, and ray-casting queries are available between most pairs of geometry types.
• Many sampling-based motion planners implemented (RRT, EST, SBL, RRT*, Lazy-RRG*, Lazy-PRM*, and more).
• Fast trajectory optimization routines.
• Real-time motion planning routines.
• Forward and inverse kinematics, forward and inverse dynamics.
• Contact mechanics computations (force closure, support polygons, stability of rigid bodies and actuated robots).
• Planning models are fully decoupled from simulation models. This helps simulate uncertainty and modeling errors.
• Robust rigid body simulation supporting triangle-soup and point cloud collisions. No need to create convex decompositions!
• Simulation of PID controlled, torque controlled, and velocity controlled motors.
• Simulation of various sensors including cameras, depth sensors, laser range finders, gyroscopes, force/torque sensors, and accelerometers.
• Works on several platforms:
  • *nux environments (x86_64, i686, Aarch64)
  • Windows
  • MacOS up to 11+
  • Google Colab

Note: newer versions of MacOS (11+) dropped OpenGL 2.0 support, so Klampt may not build. We're currently looking for alternative cross-platform graphics engines.

(Please let us know if you are able to compile on other platforms in order to help us support them in the future.)

Installation

Quick start (Python API only):

• pip install klampt (or pip3 install klampt)

To run a visualization (you may need to replace pip with pip3 and python with python3):

• pip install PyOpenGL
• pip install PyQt5
• git clone http://github.com/krishauser/Klampt-examples (this is needed to run example programs)
• cd Klampt-examples/Python3/demos
• python gl_vis.py

Installation instructions are also available for

• Linux
• Windows
• Mac OSX
• Jupyter notebook
• Docker

Klamp't works best when it is installed on your local machine, but it can also be run online through your web browser using Google Colab or Binder (or any other Jupyterhub server).

• Google Colab
• Binder

Note that the UI functionality is drastically limited compared to a local installation.

Documentation

Python Manual and API Documentation

C++ Manual

API documentation is available here

• Klamp't C++ API
• KrisLibrary C++ API

Reporting bugs and getting help

If you identify a programming bug or issue, please raise them on this Github site. If you have general questions installing or programming with Klamp't, please ask them on the Klamp't forum, which is available on GitQ: https://gitq.com/krishauser/Klampt.

Version history

master (9/10/2024)

Note: If you have a pip installed Klampt at version 0.9.2, you may get the latest updates by cloning the Git repo, then run cd Klampt/Python; python patch_a_pip_install.py. This provides all of the Python API updates listed below without needing to build from source.

• Python API: can control draw order of transparent objects with draw_order attribute.
• Python API: imageio library can be used to save movie files in the visualizer rather than relying on ffmpeg (and dumping lots of image files).
• Python API: minor bug fixes.

0.9.2 (8/13/2024)

• Python API: pip packages now built with Numpy 2+.
• Python API: Improved calibration routines in klampt.model.calibrate.
• Python API: Added surface sampling and vertex normals to klampt.model.geometry.
• Python API: Added visibility fraction determination to klampt.model.sensing.
• Python API: Added conversions of meshes to/from the trimesh library in klampt.io.trimesh_convert.
• Python API: Fixed bug in SO3/SE3 Hermite velocity interpolation. Now using extrinsic angular velocity representation as tangent vectors.
• Python API: Mouse wheel events can now be captured in visualization (Qt and GLUT backends).

0.9.1 (10/30/2023)

• Removed GLUI dependency in default build mode.
• Updated GLEW to 2.1.0.
• Builds for Linux Python 3.10+ are now working again. (Mac OS 11+ support is still TBD.)
• IK solver now has a minimization mode and a prioritized solve mode.
• Fixed problem with custom textures being unloaded after a mesh is transformed.
• Fixed bug in simulation of affine joints when the joint angle can go negative. Also, internal affine transmission coupling is simulated in a fashion that's sensitive to the driver's PID constants.
• Minor performance improvements to IK; API for getting subset of Jacobian columns.
• Python API: substantial performance improvements to camera image retrieval (camera_to_images).
• Python API: new calls for getting subset of Jacobian columns.
• Python API: Added types.convert generic utility function.
• Python API: Fixed bugs with vis module picker. Also, appearances can now have a tint set to match the C++ API.
• Python API: Bug fixes and type hints in coordinates module.
• Python API: Bug fix with cost evaluation in EmbeddedCSpace working on the embedded vectors (e.g., causing problems with outputs from makePlan)
• Python API: Removed klampt_path app and added functionality to klampt_resource app instead.
• Python API: Switch from deprecated pkg_resources to importlib for IO to resource files.
• Python API: Docs now instruct users to use pip install . rather than deprecated Setuptools.

0.9 (1/15/2022)

• Python API: type hints are available throughout the SWIG interface and in many native Python functions.
• Python API: direct Numpy interface speeds up large data copies. Tests indicate ~6x speedup (45ms->7.5ms) for getting a simulated sensor image (SimRobotSensor.getMeasurements()), ~80x speedup (160ms->2ms) for converting images to point clouds (PointCloud.setDepthImage/setRGBDImages), and 20x speedup (200ms->10ms) for converting a Numpy array to a point cloud (tested on 640x480 images).
• Python API: global functions and many classes in the the klampt.vis module now conform to PEP8 naming conventions. Old non-conformant functions produce a DeprecationWarning and will be removed in the future. (Run your code with python -W all to see these warnings.)
• Python API: major updates to the Klampt control package, including the new klampt_control utility that lets you control your robots in real time. (API-breaking change)
• Python API: standardization of robot semantic information under a RobotInfo class in klampt.model.robotinfo. Supported by klampt_control and klampt_sim. Future planners and system integration utilities will use this structure.
• Python API: klampt_resource script added which allows transfer and conversions of resources from the command line. klampt_thumbnails has been removed since all thumbnail functionality has been moved into klampt_resource.
• Python API: new system integration utilities, such as calibration (klampt.model.calibrate) and workspace calculation (klampt.model.workspace).
• Python API: textures on Appearances now fully supported.
• Python API: fixed bug with setBackgroundImage(None)
• C++ API: Everything added to the Klampt namespace. (API-breaking change!)
• C++ API: Main modeling classes renamed to align with Python API, e.g. Robot->RobotModel, RobotWorld->WorldModel, WorldSimulation->Simulator, etc. (API-breaking change!)
• Fixed bug saving/restoring simulation states.
• Some geometries support slicing and ROI (region of interest) calculations. Slicing takes a slice of a geometry with a plane, and ROI calculations determine a region of interest of the geometry. Meshes and point clouds are supported.
• Polygon and ConvexHull geometries now support ray casting.
• Projection-mapping for Appearances now supported.

0.8.7 (5/25/2021)

• Fixed bug in simulation of affine joints when the joint angle can go negative. Also, internal affine transmission coupling is simulated in a fashion that's sensitive to the driver's PID constants.
• URDF import can now import multiple collision and visual geometries.
• Python API: Workaround for Mac OSX Big Sur dropping support for OpenGL when importing PyOpenGL.
• Python API: bug fixes for motion planning with affine drivers.
• Python API: Added a function klampt.model.types.transfer() which transfers objects from one robot to another.
• Python API: Added convenience functions in klampt.model.sensing for converting camera intrinsics between Klamp't cameras and openCV and numpy formats.
• Minor bug fixes.

0.8.6 (1/27/2021)

• Fixed bug drawing Group geometries.
• RGBD sensors now have a long range for RGB information like actual cameras do, rather than clipping to the sensor's specified depth range.
• URDF mimic joints are now respected.
• Fixed bug loading non-square textures.
• Fixed bug with SubRobotModel editing in vis.
• Python API: Moved some recently added routines from klampt.model.sensor to klampt.model.geometry.
• Python API: klampt.io.ros integration with LaserScan messages and SimCameraSensor publishing.
• Python API: klampt.io.ros can now broadcast SimCameraSensor frames to ros.broadcast_tf.
• Python API: Bug fix with SimRobotSensor.kinematicSimulate and multiple worlds.
• Python API: Added visual editor for SimRobotSensor. Trajectory editor now has option to use RobotTrajectory interpolation.
• Python API: Bug fixes for HTML and IPython visualizations. vis.debug() now works for these modes, and removing text now works. Misc bugs are also fixed in the Jupyter extension.
• Python API: More thorough bounds checking in core C++ interface.
• Python API: Used Pylance to squash lots of small bugs in infrequently used branches of the code.

0.8.5 (12/7/2020)

• Fixed a few bad bugs in the pip release (klampt_browser crash; resource editor crash; HTML output with no animation; HTML output with custom entities).
• More thorough integration of custom joints into the simulation, including reading from XML world files (see Klampt-examples/data/simulation_test_worlds/jointtest.xml.
• XML world files' simulation tags can now refer to named robots, rigid objects, and terrains.
• Conversions from geometries to VolumeGrids now respect the geometry's collision margin.
• Supports emissive and specular colors in OpenGL views and XML world files now.
• The "automass" signal in .rob files and .obj (rigid object) files can now accept a parameter indicating how much mass is concentrated at the surface vs the interior of the object.
• A bug in the earlier automass inertia matrix calculation made inertias much larger than they should have been! This may require re-calibrating robot PID constants to avoid jittering. (This bug will not affect robot / object files that specify inertias matrices.)
• Python API: Can now remesh triangle meshes, point clouds, and volume grids using Geometry3D.convert(self.type(),resolution).
• Python API: Easy vis.snapshot API added.

0.8.3 (10/24/2020)

• Convex hulls are a new geometry type. They support conversion from multiple geometry types and conversion to meshes, visualization, and fast convex hull-convex hull proximity queries. (Queries are not yet supported for convex hull vs anything else.)
• Python API: Python 3 is now the default version used by Klamp't, and all further feature development will be focused on Python 3. The Python 2 version will receive occasional bug fixes but will be deprecated by the next version.
• Python API: The visualization API has a unified interface between PyQT, PyOpenGL, Jupyter notebooks, and HTML output. This makes it easy to convert your OpenGL visualizations into shareable, animated web pages (with some caveats). Use vis.init() to select which visualization backend to use.
• Python API: A new klampt.math.autodiff module supports taking derivatives of many Klamp't functions. This lets you use optimization for, say, trajectory optimization, deep learning, etc. A Pytorch binding is supported.
• Python API: A new kinematic trajectory optimization toolbox is available, klampt.plan.kinetrajopt. This uses convex hulls to enforce collision constraints along a path.
• Python API: A new colorization module, klampt.vis.colorize, makes it easy to color-code point clouds and meshes for classy visualizations.
• Python API: The klampt.math.vis module now supports a debug function for one-line visualization, and a followCamera function that is helpful for making animations.
• Python API: The Jupyter / HTML API backends are upgraded to use the latest version of three.js.
• Python API: The klampt.math.symbolic now supports tensor derivatives (e.g., the derivative of vector w.r.t. a matrix).
• Python API: the RobotModel class now exposes reduce and mount methods.
• Python API: The IKObjective, Geometry3D, GeometricPrimitive, TriangleMesh, PointCloud, VolumeGrid, and ConvexHull classes can now be pickled. Geometry objects now have JSON loaders/savers.
• Python API ray casting can now return an index of the triangle / point that was hit: Geometry3D.rayCast_ext. VolumeGrids now support ray casting.
• Python API: Trajectory functionality has been enhanced. Can now do acceleration-bounded and velocity-bounded minimum-time interpolation, smoothed (Hermite-like) geodesic interpolation on manifolds. Hermite, SO(3), and SE(3) trajectories can now be saved/loaded to/from JSON.
• Python API: [EXPERIMENTAL] Klamp't can now drive your robots with the new Python klampt.control module! The RobotInterfaceBase class provides a unified interface for working with real motor controllers. For example, if you build a robot with basic position-controlled servos, the RobotInterfaceCompleter class provides a unified interface for executing trajectories on servos, motion smoothing, emergency stops, and Cartesian control. Also, if you build a robot consisting of multiple parts, such as an arm and a gripper, you can perform coordinated control using the MultiRobotInterface class.
• Python API: Added SimJoint class to enable creation of sliding / hinge / fixed joints between bodies in the simulator.
• C++ API: Robot class and the Python RobotModel class have methods to reduce the number of DOFs of a robot that contains fixed DOFs.
• Miscellaneous bug fixes, improvements to the docs.

0.8.2 (11/15/2019)

• Can now load almost all files from URLs! Just use an http:// prefix to the filenames of worlds, robots, rigid objects, or geometries. This also works for resources in Python. (Resources in C++ are not done yet)
• New mesh visualization shows smooth meshes and black silhouettes by default, making things look prettier.
• Can now mount robots in URDF files (see tutorial for an example).
• Added ability to specify cost functions in kinematic planning.
• Added conversions to Numpy objects in Python API (klampt.io.numpy_convert)
• Added much more ROS support in Python API (klampt.io.ros module).
• Added conversions to Open3D objects in Python API (klampt.io.open3d_convert)
• The Python loading API has changed to have a more complete list of types and file extensions.
• Fixed some bugs when editing in the klampt_browser Python program.
• Fixed some motion planning bugs (closed loop + moving subset spaces in Python, some planners in C++).
• Fixed a bizarre Python 2 to 3 conversion bug in se3.mul.

0.8.1 (1/3/2019)

• Cleaned up documentation, separating C++ and Python docs. Sphinx is now used for Python docs.
• Added klampt_path utility to Python API.
• Added Python API bindings for higher order dynamics functions that were already in the C++ library (Cartesian accelerations, Hessians, derivatives of mass matrices).
• Added rootfind Python C++ extension back.

0.8.0 (12/17/2018)

• Cleaner file structure, with C++ files in the Cpp directory.
• Improved build system for Python, allowing easy installation via pip install python.
• Integration with Jupyter Notebook
• Added Python utility programs (klampt_browse, klampt_sim, klampt_thumbnail).
• Improvements support Python visualization on Mac.
• Upgraded to PyQt5. PyQt4 is still supported for now.
• Geometry conversions exposed in Python via the convert function.
• Improved usage of some graphics card resources for streaming point clouds.
• Support for LOG4CXX logging.
• Removed dependencies on Boost and upgraded to C++11.
• Removed dependencies on GLUT and GLUI. (Some examples still need to be upgraded to Qt.)
• Cleaned up some cruft in KrisLibrary.

0.7 (3/24/2017)

• Improved simulation stability, including adaptive time stepping and instability detection/recovery.
• The proprietary .tri geometry file format has been replaced with the Object File Format (OFF) for better compatibility with 3D modeling packages.
• Simulated visual, depth, and laser range sensors are now fully supported.
• ROS sensor simulation broadcasting is enabled in Klampt/IO/ROS.h.
• World XML files can now be saved to disk.
• Robot sensors and controllers can be attached directly to a robot model using the sensors / controller properties in the robot's .rob or .urdf file.
• The motion planning structure in KrisLibrary has been completely revamped in preparation for support of optimal and kinodynamic planning, but this should be a mostly transparent change to Klamp't users.
• The Python interface is now better organized. However, the module structure is incompatible with code developed for versions 0.6.2 and earlier. In particular, math modules (vectorops, so3, se3) are now in the math subpackage, and visualization modules (glprogram, glrobotprogram, etc) are now in the vis subpackage.
• Custom Python simulations of sensors, actuators, and force appliers that work on fast simulation rates are easier to integrate with slower control loops in the sim.simulation module.
• Revamped and enhanced Python visualization functionality in the vis module. Multiple windows, simultaneous viewports, trajectory visualization, custom in-visualization plotting, automatic viewport determination, and thumbnail saving are now supported.
• Cartesian trajectory generation, file loading utilities are added to Python.

0.6.2 (7/31/2016)

• New Python APIs for visualization
• Geometry caching helps load times and memory usage for large scenes
• A global IK solver has been added to the Python API
• ROS broadcasting / subscribing is enabled in the C++ API.

0.6.1 (3/21/2015)

• Added functionality in Python API to load/save/edit resources, manipulate transforms and robot configurations via widgets, change appearance of objects, and run programs through Qt.
• Removed the Python collide module. All prior functionality is now placed in the Geometry3D class in the standard klampt module.
• Real-time planning interface has been greatly simplified.
• The MilestonePathController class will be deprecated, use PolynomialPathController instead.
• Minor bug fixes

0.6 (7/4/2014)

• CMake build system makes it easier to build across multiple platforms
• Easy connections with external controllers via ROS or a serial protocol
• More user-friendly Qt application front ends
• More demos, example code, and tutorials
• Direct loading of URDF files with <klampt> XML tag
• More calibrated robots: Baxter, RobotiQ 3-finger adaptive gripper
• Unification of locomotion and manipulation via the GeneralizedRobot mechanism
• Fixed build for Cygwin
• More sophisticated logging capabilities in SimTest (contacts, commanded/actual/sensed paths)
• Miscellaneous debugging throughout

0.5. Initial release (11/17/2013)

Who uses Klamp't?

(This is not an exhaustive list; if you are using Klampt and would like to be listed, let us know!)

• Intelligent Motion Laboratory (IML), on the Baxter, TRINA, RoboSimian, and UR5 robots. Klamp't has also been used in dozens of academic papers.

• Multiple IML open-source projects, including:

  • Global Redundancy Resolution
  • Semi-Infinite Optimization
  • Inverse Kinematics Database (IKDB)
  • Robotic Systems Open Textbook

• Courses at UIUC (CS 498) and Duke (ECE 383, ECE 489)

• Team Duke in the Amazon Picking Challenge 2016-2018.

• SimGrasp at Stanford University

• Research labs at Brown, RPI, Columbia, and IIT Pisa

• TeamHubo in the DARPA Robotics Challenge

• IROS 2016 Manipulation Challenge, Simulation Track

Comparison to related packages

• ROS (Robot Operating System) is a middleware system designed for distributed control of physical robots, and Klamp't is designed to be interoperable with it. Various ROS software packages can replicate many of the functions of Klamp't when used together (Gazebo, KDE, Rviz, MoveIt!), but this approach is difficult since these tools are not as tightly integrated as they are in Klamp't. ROS has limited support for legged robots, and is poorly suited for prototyping high-rate feedback control systems. ROS is heavy-weight, has a steep learning curve especially for non-CS students, and is also not completely cross-platform (only Ubuntu is fully supported).
• OpenRAVE (Robotics and Animation Virtual Environment) is similar to Klamp't and was developed concurrently by a similar group at CMU. OpenRAVE has more sophisticated manipulation planning functionality. Does not support planning for legged robots, but simulation is possible with some effort. Simulation models are often conflated with planning models whereas in Klamp't they are fully decoupled. OpenRAVE is no longer actively supported.
• Gazebo, Webots, V-REP, etc are robot simulation packages built off of the same class of rigid body simulations as Klamp't. They have more sophisticated sensor simulation capabilities, cleaner APIs, and nicer visualizations but are typically built for mobile robots and have limited functionality for modeling, planning, and optimization. Klamp't also has improved mesh-mesh collision handling that makes collision handling much more stable.

Contributors

Kris Hauser has been the primary maintainer throughout the project. Other major contributors include Zherong Pan, Gao Tang, Jordan Tritell, Jingru Luo, and Alessio Rocchi.

Adam Konnecker, Cam Allen, and Steve Kuznetsov have helped with the Mac build. Hayden Bader helped with the prebuilt Docker container.

As an open-source project, we welcome contributions and suggestions from the community.