pydoit.org

What It Is

pydoit is a Python-native task runner and automation tool, positioned somewhat like make, but without Makefiles, DSLs, or YAML. Instead, tasks are defined with Python dicts and functions. It determines whether a task needs to be rerun based on file dependencies, targets, MD5 hashes, or timestamps, and supports DAG execution. Its core value is therefore: “only run the parts that have changed.”

Core Capabilities

Functionally, doit supports shell commands, Python functions, or a mix of both as task actions. Tasks can be self-documented via docstrings and inspected with doit list and doit help. Its incremental build capabilities are particularly strong, with support for file_dep, targets, custom up-to-date checks, calc_dep, result validation, and dynamic task generation, making it suitable for complex pipelines. Advanced features include a plugin architecture, Framework API, custom commands, storage backends, task loaders, output reporters, thread- or process-based parallelism, bash/zsh completion, and Graphviz DAG visualization.

Ecosystem and Documentation

The project requires Python 3.10+, uses the MIT License, is published on PyPI, and accepts contributions on GitHub. The main text indicates that it has been continuously maintained since 2008, with 99% unit test coverage. Its documentation is fairly comprehensive, covering getting started guides, tutorials, references, CLI usage, configuration, extensions, FAQ, and integration tools. Success stories include Stanford, BMW, Atomwise, McGill, MetalK8s, Nikola, and the University of Oslo Library, suggesting that it is not just a toy tool but is used in research, build systems, content generation, and DevOps scenarios.

Pricing and Open Source

The text does not mention a commercial edition, subscription, or paid support. Given the MIT License and installation via pip install doit, it can be regarded as a free and open-source tool. It is a local CLI/library rather than a SaaS product, so there is no conventional self-hosting deployment issue; it can be run directly locally, in a repository, or in a CI environment.

Pros, Cons, and Best Fit

Its strengths are that it is lightweight, Pythonic, testable, and debuggable, with stronger dependency tracking than command execution tools such as invoke. Compared with platforms like Airflow or Luigi, it avoids the overhead of a distributed scheduling framework. Its drawbacks are that it mainly serves the Python ecosystem, and teams need to understand doit’s task model. If you need large-scale distributed scheduling, a Web UI, or enterprise SLAs, the source text does not indicate that these capabilities are available. It is well suited to Python developers, research data pipelines, build system maintainers, static site generation, game asset pipelines, and DevOps release automation.

Access from China

The text does not provide information about mainland China network access, mirrors, or payments. Since it is available via PyPI/GitHub, actual access may depend on the local network environment. Alternatives to consider include make, just, invoke, snakemake, Scons, waf, Luigi, Airflow, or CMake.