diff --git a/.git-blame-ignore-revs b/.git-blame-ignore-revs index 690e5f46a4fc0..195e1e06eea46 100644 --- a/.git-blame-ignore-revs +++ b/.git-blame-ignore-revs @@ -1,3 +1,12 @@ +# all: Prune trailing whitespace. +dda9b9c6da5d3c31fa8769e581a753e95a270803 + +# all: Remove the "STATIC" macro and just use "static" instead. +decf8e6a8bb940d5829ca3296790631fcece7b21 + +# renesas-ra: Fix spelling mistakes found by codespell. +b3f2f18f927fa2fad10daf63d8c391331f5edf58 + # all: Update Python formatting to ruff-format. bbd8760bd9a2302e5abee29db279102bb11d7732 diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md deleted file mode 100644 index 7bad9562964af..0000000000000 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -name: Bug report -about: Report an issue -title: '' -labels: bug -assignees: '' - ---- - -* Please search existing issues before raising a new issue. For questions about MicroPython or for help using MicroPython, or any sort of "how do I?" requests, please use the Discussions tab or raise a documentation request instead. - -* In your issue, please include a clear and concise description of what the bug is, the expected output, and how to replicate it. - -* If this issue involves external hardware, please include links to relevant datasheets and schematics. - -* If you are seeing code being executed incorrectly, please provide a minimal example and expected output (e.g. comparison to CPython). - -* For build issues, please include full details of your environment, compiler versions, command lines, and build output. - -* Please provide as much information as possible about the version of MicroPython you're running, such as: - - firmware file name - - git commit hash and port/board - - version information shown in the REPL (hit Ctrl-B to see the startup message) - -* Remove all placeholder text above before submitting. diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml new file mode 100644 index 0000000000000..1ec6c7067f9cd --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -0,0 +1,109 @@ +name: Bug report +description: Report a bug or unexpected behaviour +labels: ["bug"] +body: + - type: markdown + attributes: + value: | + Please provide as much detail as you can, it really helps us find and fix bugs faster. + + #### Not a bug report? + + * If you have a question \"How Do I ...?\", please post it on [GitHub Discussions](https://github.com/orgs/micropython/discussions/) or [Discord](https://discord.gg/RB8HZSAExQ) instead of here. + * For missing or incorrect documentation, or feature requests, then please [choose a different issue type](https://github.com/micropython/micropython/issues/new/choose). + + #### Existing issue? + + * Please search for [existing issues](https://github.com/micropython/micropython/issues) matching this bug before reporting. + - type: input + id: port-board-hw + attributes: + label: Port, board and/or hardware + description: | + Which MicroPython port(s) and board(s) are you using? + placeholder: | + esp32 port, ESP32-Fantastic board. + validations: + required: true + - type: textarea + id: version + attributes: + label: MicroPython version + description: | + To find the version: + + 1. Open a serial REPL. + 2. Type Ctrl-B to see the startup message. + 3. Copy-paste that output here. + + If the issue is about building MicroPython, please provide output of `git describe --dirty` and as much information as possible about the build environment. + + If the version or configuration is modified from the official MicroPython releases or the master branch, please tell us the details of this as well. + placeholder: | + MicroPython v6.28.3 on 2029-01-23; PyBoard 9 with STM32F9 + validations: + required: true + - type: textarea + id: steps-reproduce + attributes: + label: Reproduction + description: | + What steps will reproduce the problem? Please include all details that could be relevant about the environment, configuration, etc. + + If there is Python code to reproduce this issue then please either: + a. Type it into a code block below ([code block guide](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/creating-and-highlighting-code-blocks)), or + b. Post longer code to a [GitHub gist](https://gist.github.com/), or + c. Create a sample project on GitHub. + + For build issues, please provide the exact build commands that you ran. + placeholder: | + 1. Copy paste the code provided below into a new file + 2. Use `mpremote run` to execute it on the board. + validations: + required: true + - type: textarea + id: expected + attributes: + label: Expected behaviour + description: | + What did you expect MicroPython to do? If comparing output with CPython or a different MicroPython port/version then please provide that output here. + placeholder: | + Expected to print "Hello World". + + Here is the correct output, seen with previous MicroPython version v3.14.159: + + > [...] + - type: textarea + id: what-happened + attributes: + label: Observed behaviour + description: | + What actually happened? Where possible please paste exact output, or the complete build log, etc. Very long output can be linked in a [GitHub gist](https://gist.github.com/). + placeholder: | + This unexpected exception appears: + + > [...] + validations: + required: true + - type: textarea + id: additional + attributes: + label: Additional Information + description: | + Is there anything else that might help to resolve this issue? + value: No, I've provided everything above. + - type: dropdown + id: code-of-conduct + attributes: + label: Code of Conduct + description: | + Do you agree to follow the MicroPython [Code of Conduct](https://github.com/micropython/micropython/blob/master/CODEOFCONDUCT.md) to ensure a safe and respectful space for everyone? + options: + - "Yes, I agree" + multiple: true + validations: + required: true + - type: markdown + attributes: + value: | + Thanks for taking the time to help improve MicroPython. diff --git a/.github/ISSUE_TEMPLATE/documentation.md b/.github/ISSUE_TEMPLATE/documentation.md deleted file mode 100644 index e36fa62ac29a4..0000000000000 --- a/.github/ISSUE_TEMPLATE/documentation.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -name: Documentation issue -about: Report areas of the documentation or examples that need improvement -title: 'docs: ' -labels: documentation -assignees: '' - ---- - -* Please search existing issues before raising a new issue. For questions about MicroPython or for help using MicroPython, or any sort of "how do I?" requests, please use the Discussions tab instead. - -* Describe what was missing from the documentation and/or what was incorrect/incomplete. - -* If possible, please link to the relevant page on https://docs.micropython.org/ - -* Remove all placeholder text above before submitting. diff --git a/.github/ISSUE_TEMPLATE/documentation.yml b/.github/ISSUE_TEMPLATE/documentation.yml new file mode 100644 index 0000000000000..93051e51c80c7 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/documentation.yml @@ -0,0 +1,46 @@ +name: Documentation issue +description: Report areas of the documentation or examples that need improvement +title: "docs: " +labels: ["documentation"] +body: + - type: markdown + attributes: + value: | + This form is for reporting issues with the documentation or examples provided with MicroPython. + + If you have a general question \"How Do I ...?\", please post it on [GitHub Discussions](https://github.com/orgs/micropython/discussions/) or [Discord](https://discord.gg/RB8HZSAExQ) instead of here. + + #### Existing issue? + + * Please search for [existing issues](https://github.com/micropython/micropython/issues) before reporting a new one. + - type: input + id: page + attributes: + label: Documentation URL + description: | + Does this issue relate to a particular page in the [online documentation](https://docs.micropython.org/en/latest/)? If yes, please paste the URL of the page: + placeholder: | + https://docs.micropython.org/en/latest/ + - type: textarea + id: version + attributes: + label: Description + description: | + Please describe what was missing from the documentation and/or what was incorrect/incomplete. + validations: + required: true + - type: dropdown + id: code-of-conduct + attributes: + label: Code of Conduct + description: | + Do you agree to follow the MicroPython [Code of Conduct](https://github.com/micropython/micropython/blob/master/CODEOFCONDUCT.md) to ensure a safe and respectful space for everyone? + options: + - "Yes, I agree" + multiple: true + validations: + required: true + - type: markdown + attributes: + value: | + Thanks for taking the time to help improve MicroPython. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md deleted file mode 100644 index 81b55d98e0da8..0000000000000 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -name: Feature request -about: Request a feature or improvement -title: '' -labels: enhancement -assignees: '' - ---- - -* Please search existing issues before raising a new issue. For questions about MicroPython or for help using MicroPython, or any sort of "how do I?" requests, please use the Discussions tab or raise a documentation request instead. - -* Describe the feature you'd like to see added to MicroPython. In particular, what does this feature enable and why is it useful. MicroPython aims to strike a balance between functionality and code size, so please consider whether this feature can be optionally enabled and whether it can be provided in other ways (e.g. pure-Python library). - -* For core Python features, where possible please include a link to the relevant PEP. - -* For new architectures / ports / boards, please provide links to relevant documentation, specifications, and toolchains. Any information about the popularity and unique features about this hardware would also be useful. - -* For features for existing ports (e.g. new peripherals or microcontroller features), please describe which port(s) it applies too, and whether this is could be an extension to the machine API or a port-specific module? - -* For drivers (e.g. for external hardware), please link to datasheets and/or existing drivers from other sources. - -* Who do you expect will implement the feature you are requesting? Would you be willing to sponsor this work? - -* Remove all placeholder text above before submitting. diff --git a/.github/ISSUE_TEMPLATE/feature_request.yml b/.github/ISSUE_TEMPLATE/feature_request.yml new file mode 100644 index 0000000000000..7d5162a32a4af --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.yml @@ -0,0 +1,74 @@ +name: Feature request +description: Request a feature or improvement +labels: ['enhancement'] +body: + - type: markdown + attributes: + value: | + This form is for requesting features or improvements in MicroPython. + + #### Get feedback first + + Before submitting a new feature idea here, suggest starting a discussion on [Discord](https://discord.gg/RB8HZSAExQ) or [GitHub Discussions](https://github.com/orgs/micropython/discussions/) to get early feedback from the community and maintainers. + + #### Not a MicroPython core feature? + + * If you have a question \"How Do I ...?\", please post it on GitHub Discussions or Discord instead of here. + * Could this feature be implemented as a pure Python library? If so, please open the request on the [micropython-lib repository](https://github.com/micropython/micropython-lib/issues) instead. + + #### Existing issue? + + * Please search for [existing issues](https://github.com/micropython/micropython/issues) before opening a new one. + - type: textarea + id: feature + attributes: + label: Description + description: | + Describe the feature you'd like to see added to MicroPython. What does this feature enable and why is it useful? + + * For core Python features, where possible please include a link to the relevant PEP or CPython documentation. + * For new architectures / ports / boards, please provide links to relevant documentation, specifications, and toolchains. Any information about the popularity and unique features about this hardware would also be useful. + * For features for existing ports (e.g. new peripherals or microcontroller features), please describe which port(s) it applies to, and whether this is could be an extension to the machine API or a port-specific module? + * For drivers (e.g. for external hardware), please link to datasheets and/or existing drivers from other sources. + + If there is an existing discussion somewhere about this feature, please add a link to it as well. + validations: + required: true + - type: textarea + id: size + attributes: + label: Code Size + description: | + MicroPython aims to strike a balance between functionality and code size. Can this feature be optionally enabled? + + If you believe the usefulness of this feature would outweigh the additional code size, please explain. (It's OK to say you're unsure here, we're happy to discuss this with you.) + - type: dropdown + id: implementation + attributes: + label: Implementation + description: | + What is your suggestion for implementing this feature? + + (See also: [How to sponsor](https://github.com/sponsors/micropython#sponsors), [How to submit a Pull Request](https://github.com/micropython/micropython/wiki/ContributorGuidelines).) + options: + - I hope the MicroPython maintainers or community will implement this feature + - I intend to implement this feature and would submit a Pull Request if desirable + - I would like to sponsor development of this feature + multiple: true + validations: + required: true + - type: dropdown + id: code-of-conduct + attributes: + label: Code of Conduct + description: | + Do you agree to follow the MicroPython [Code of Conduct](https://github.com/micropython/micropython/blob/master/CODEOFCONDUCT.md) to ensure a safe and respectful space for everyone? + options: + - "Yes, I agree" + multiple: true + validations: + required: true + - type: markdown + attributes: + value: | + Thanks for taking the time to suggest improvements for MicroPython. diff --git a/.github/ISSUE_TEMPLATE/security.md b/.github/ISSUE_TEMPLATE/security.md deleted file mode 100644 index cfe4a4befdb14..0000000000000 --- a/.github/ISSUE_TEMPLATE/security.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -name: Security report -about: Report a security issue or vulnerability in MicroPython -title: '' -labels: security -assignees: '' - ---- - -* If you need to raise this issue privately with the MicroPython team, please email contact@micropython.org instead. - -* Include a clear and concise description of what the security issue is. - -* What does this issue allow an attacker to do? - -* Remove all placeholder text above before submitting. diff --git a/.github/ISSUE_TEMPLATE/security.yml b/.github/ISSUE_TEMPLATE/security.yml new file mode 100644 index 0000000000000..57c2a5885ed85 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/security.yml @@ -0,0 +1,60 @@ +name: Security report +description: Report a security issue or vulnerability in MicroPython +labels: ["security"] +body: + - type: markdown + attributes: + value: | + This form is for reporting security issues in MicroPython that are not readily exploitable. + + 1. For issues that are readily exploitable or have high impact, please email contact@micropython.org instead. + 1. If this is a question about security, please ask it in [Discussions](https://github.com/orgs/micropython/discussions/) or [Discord](https://discord.gg/RB8HZSAExQ) instead. + + #### Existing issue? + + * Please search for [existing issues](https://github.com/micropython/micropython/issues) before reporting a new one. + + - type: input + id: port-board-hw + attributes: + label: Port, board and/or hardware + description: | + Which MicroPython port(s) and board(s) are you using? + placeholder: | + esp32 port, ESP32-Duper board. + - type: textarea + id: version + attributes: + label: MicroPython version + description: | + To find the version: + + 1. Open a serial REPL. + 2. Type Ctrl-B to see the startup message. + 3. Copy-paste that output here. + + If the version or configuration is modified from the official MicroPython releases or the master branch, please tell us the details of this as well. + placeholder: | + MicroPython v6.28.3 on 2029-01-23; PyBoard 9 with STM32F9 + - type: textarea + id: report + attributes: + label: Issue Report + description: | + Please provide a clear and concise description of the security issue. + + * What does this issue allow an attacker to do? + * How does the attacker exploit this issue? + validations: + required: true + - type: dropdown + id: code-of-conduct + attributes: + label: Code of Conduct + description: | + Do you agree to follow the MicroPython [Code of Conduct](https://github.com/micropython/micropython/blob/master/CODEOFCONDUCT.md) to ensure a safe and respectful space for everyone? + options: + - "Yes, I agree" + multiple: true + validations: + required: true diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 0000000000000..e11cebddb3710 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,33 @@ + + +### Summary + + + + +### Testing + + + + +### Trade-offs and Alternatives + + + diff --git a/.github/workflows/biome.yml b/.github/workflows/biome.yml new file mode 100644 index 0000000000000..88744f16ca7d6 --- /dev/null +++ b/.github/workflows/biome.yml @@ -0,0 +1,16 @@ +name: JavaScript code lint and formatting with Biome + +on: [push, pull_request] + +jobs: + eslint: + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + - name: Setup Biome + uses: biomejs/setup-biome@v2 + with: + version: 1.5.3 + - name: Run Biome + run: biome ci --indent-style=space --indent-width=4 tests/ ports/webassembly diff --git a/.github/workflows/code_formatting.yml b/.github/workflows/code_formatting.yml index 5cd7984f7ac15..9f30f048cfdbe 100644 --- a/.github/workflows/code_formatting.yml +++ b/.github/workflows/code_formatting.yml @@ -18,13 +18,3 @@ jobs: run: source tools/ci.sh && ci_c_code_formatting_run - name: Check code formatting run: git diff --exit-code - - code-spelling: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: actions/setup-python@v5 - - name: Install packages - run: source tools/ci.sh && ci_code_spell_setup - - name: Run spell checker - run: source tools/ci.sh && ci_code_spell_run diff --git a/.github/workflows/code_size.yml b/.github/workflows/code_size.yml index a65a00201a7d2..67261933798cb 100644 --- a/.github/workflows/code_size.yml +++ b/.github/workflows/code_size.yml @@ -1,16 +1,21 @@ name: Check code size on: - push: pull_request: paths: - '.github/workflows/*.yml' - 'tools/**' - 'py/**' - 'extmod/**' + - 'shared/**' - 'lib/**' - 'ports/bare-arm/**' + - 'ports/mimxrt/**' - 'ports/minimal/**' + - 'ports/rp2/**' + - 'ports/samd/**' + - 'ports/stm32/**' + - 'ports/unix/**' concurrency: group: ${{ github.workflow }}-${{ github.ref }} @@ -18,7 +23,7 @@ concurrency: jobs: build: - runs-on: ubuntu-20.04 + runs-on: ubuntu-22.04 steps: - uses: actions/checkout@v4 with: diff --git a/.github/workflows/code_size_comment.yml b/.github/workflows/code_size_comment.yml index a394f7a14b054..521afad709d17 100644 --- a/.github/workflows/code_size_comment.yml +++ b/.github/workflows/code_size_comment.yml @@ -11,7 +11,7 @@ concurrency: jobs: comment: - runs-on: ubuntu-20.04 + runs-on: ubuntu-22.04 steps: - name: 'Download artifact' id: download-artifact diff --git a/.github/workflows/codespell.yml b/.github/workflows/codespell.yml new file mode 100644 index 0000000000000..1d6b1dc9d2b27 --- /dev/null +++ b/.github/workflows/codespell.yml @@ -0,0 +1,13 @@ +name: Check spelling with codespell + +on: [push, pull_request] + +jobs: + codespell: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + # codespell version should be kept in sync with .pre-commit-config.yml + - run: pip install --user codespell==2.4.1 tomli + - run: codespell + diff --git a/.github/workflows/commit_formatting.yml b/.github/workflows/commit_formatting.yml index 3fdcabc4ca73f..fcbcaa7092ee2 100644 --- a/.github/workflows/commit_formatting.yml +++ b/.github/workflows/commit_formatting.yml @@ -1,6 +1,6 @@ name: Check commit message formatting -on: [push, pull_request] +on: [pull_request] concurrency: group: ${{ github.workflow }}-${{ github.ref }} @@ -12,7 +12,7 @@ jobs: steps: - uses: actions/checkout@v4 with: - fetch-depth: '100' + fetch-depth: 100 - uses: actions/setup-python@v5 - name: Check commit message formatting run: source tools/ci.sh && ci_commit_formatting_run diff --git a/.github/workflows/mpy_format.yml b/.github/workflows/mpy_format.yml index baa02ce08d507..b6768a46c3cdc 100644 --- a/.github/workflows/mpy_format.yml +++ b/.github/workflows/mpy_format.yml @@ -15,7 +15,7 @@ concurrency: jobs: test: - runs-on: ubuntu-20.04 # use 20.04 to get python2 + runs-on: ubuntu-22.04 # use 22.04 to get python2 steps: - uses: actions/checkout@v4 - name: Install packages diff --git a/.github/workflows/ports_qemu-arm.yml b/.github/workflows/ports_alif.yml similarity index 51% rename from .github/workflows/ports_qemu-arm.yml rename to .github/workflows/ports_alif.yml index db3cd7871d591..0e96e7d816e50 100644 --- a/.github/workflows/ports_qemu-arm.yml +++ b/.github/workflows/ports_alif.yml @@ -1,4 +1,4 @@ -name: qemu-arm port +name: alif port on: push: @@ -11,22 +11,23 @@ on: - 'shared/**' - 'lib/**' - 'drivers/**' - - 'ports/qemu-arm/**' - - 'tests/**' + - 'ports/alif/**' concurrency: group: ${{ github.workflow }}-${{ github.ref }} cancel-in-progress: true jobs: - build_and_test: + build_alif: + strategy: + fail-fast: false + matrix: + ci_func: # names are functions in ci.sh + - alif_ae3_build runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Install packages - run: source tools/ci.sh && ci_qemu_arm_setup - - name: Build and run test suite - run: source tools/ci.sh && ci_qemu_arm_build - - name: Print failures - if: failure() - run: grep --before-context=100 --text "FAIL" ports/qemu-arm/build/console.out + run: source tools/ci.sh && ci_alif_setup + - name: Build ci_${{matrix.ci_func }} + run: source tools/ci.sh && ci_${{ matrix.ci_func }} diff --git a/.github/workflows/ports_esp32.yml b/.github/workflows/ports_esp32.yml index 6872b9d12de78..4c07f89437757 100644 --- a/.github/workflows/ports_esp32.yml +++ b/.github/workflows/ports_esp32.yml @@ -25,17 +25,17 @@ jobs: ci_func: # names are functions in ci.sh - esp32_build_cmod_spiram_s2 - esp32_build_s3_c3 - runs-on: ubuntu-20.04 + runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - id: idf_ver - name: Read the ESP-IDF version - run: source tools/ci.sh && echo "IDF_VER=$IDF_VER" | tee "$GITHUB_OUTPUT" + name: Read the ESP-IDF version (including Python version) + run: source tools/ci.sh && echo "IDF_VER=${IDF_VER}-py${PYTHON_VER}" | tee "$GITHUB_OUTPUT" - name: Cached ESP-IDF install id: cache_esp_idf - uses: actions/cache@v3 + uses: actions/cache@v4 with: path: | ./esp-idf/ diff --git a/.github/workflows/ports_mimxrt.yml b/.github/workflows/ports_mimxrt.yml index 614d7458079e8..7743e036ab377 100644 --- a/.github/workflows/ports_mimxrt.yml +++ b/.github/workflows/ports_mimxrt.yml @@ -19,9 +19,14 @@ concurrency: jobs: build: - runs-on: ubuntu-20.04 + runs-on: ubuntu-latest + defaults: + run: + working-directory: 'micropython repo' # test build with space in path steps: - uses: actions/checkout@v4 + with: + path: 'micropython repo' - name: Install packages run: source tools/ci.sh && ci_mimxrt_setup - name: Build diff --git a/.github/workflows/ports_nrf.yml b/.github/workflows/ports_nrf.yml index d9cffb9778cba..76727c9d1f6bd 100644 --- a/.github/workflows/ports_nrf.yml +++ b/.github/workflows/ports_nrf.yml @@ -19,7 +19,7 @@ concurrency: jobs: build: - runs-on: ubuntu-20.04 + runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Install packages diff --git a/.github/workflows/ports_qemu.yml b/.github/workflows/ports_qemu.yml new file mode 100644 index 0000000000000..ac09dde86408b --- /dev/null +++ b/.github/workflows/ports_qemu.yml @@ -0,0 +1,51 @@ +name: qemu port + +on: + push: + pull_request: + paths: + - '.github/workflows/*.yml' + - 'tools/**' + - 'py/**' + - 'extmod/**' + - 'shared/**' + - 'lib/**' + - 'drivers/**' + - 'ports/qemu/**' + - 'tests/**' + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + build_and_test_arm: + strategy: + fail-fast: false + matrix: + ci_func: # names are functions in ci.sh + - bigendian + - sabrelite + - thumb + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Install packages + run: source tools/ci.sh && ci_qemu_setup_arm + - name: Build and run test suite ci_qemu_build_arm_${{ matrix.ci_func }} + run: source tools/ci.sh && ci_qemu_build_arm_${{ matrix.ci_func }} + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures + + build_and_test_rv32: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Install packages + run: source tools/ci.sh && ci_qemu_setup_rv32 + - name: Build and run test suite + run: source tools/ci.sh && ci_qemu_build_rv32 + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures diff --git a/.github/workflows/ports_renesas-ra.yml b/.github/workflows/ports_renesas-ra.yml index b1a30c2f11798..b9fa74331dc0b 100644 --- a/.github/workflows/ports_renesas-ra.yml +++ b/.github/workflows/ports_renesas-ra.yml @@ -19,7 +19,7 @@ concurrency: jobs: build_renesas_ra_board: - runs-on: ubuntu-20.04 + runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Install packages diff --git a/.github/workflows/ports_rp2.yml b/.github/workflows/ports_rp2.yml index 5741b75d2aa79..748f38e143893 100644 --- a/.github/workflows/ports_rp2.yml +++ b/.github/workflows/ports_rp2.yml @@ -20,8 +20,13 @@ concurrency: jobs: build: runs-on: ubuntu-latest + defaults: + run: + working-directory: 'micropython repo' # test build with space in path steps: - uses: actions/checkout@v4 + with: + path: 'micropython repo' - name: Install packages run: source tools/ci.sh && ci_rp2_setup - name: Build diff --git a/.github/workflows/ports_stm32.yml b/.github/workflows/ports_stm32.yml index 84d30b27f6e66..8800f145189b8 100644 --- a/.github/workflows/ports_stm32.yml +++ b/.github/workflows/ports_stm32.yml @@ -25,7 +25,8 @@ jobs: ci_func: # names are functions in ci.sh - stm32_pyb_build - stm32_nucleo_build - runs-on: ubuntu-20.04 + - stm32_misc_build + runs-on: ubuntu-22.04 steps: - uses: actions/checkout@v4 - name: Install packages diff --git a/.github/workflows/ports_unix.yml b/.github/workflows/ports_unix.yml index 2367eddbe6ea5..662121654e4f7 100644 --- a/.github/workflows/ports_unix.yml +++ b/.github/workflows/ports_unix.yml @@ -88,16 +88,17 @@ jobs: (cd ports/unix && gcov -o build-coverage/py ../../py/*.c || true) (cd ports/unix && gcov -o build-coverage/extmod ../../extmod/*.c || true) - name: Upload coverage to Codecov - uses: codecov/codecov-action@v3 + uses: codecov/codecov-action@v5 with: fail_ci_if_error: true verbose: true + token: ${{ secrets.CODECOV_TOKEN }} - name: Print failures if: failure() run: tests/run-tests.py --print-failures coverage_32bit: - runs-on: ubuntu-20.04 # use 20.04 to get libffi-dev:i386 + runs-on: ubuntu-22.04 # use 22.04 to get libffi-dev:i386 steps: - uses: actions/checkout@v4 - name: Install packages @@ -115,7 +116,7 @@ jobs: run: tests/run-tests.py --print-failures nanbox: - runs-on: ubuntu-20.04 # use 20.04 to get python2, and libffi-dev:i386 + runs-on: ubuntu-22.04 # use 22.04 to get python2, and libffi-dev:i386 steps: - uses: actions/checkout@v4 - name: Install packages @@ -141,7 +142,7 @@ jobs: run: tests/run-tests.py --print-failures stackless_clang: - runs-on: ubuntu-20.04 + runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Install packages @@ -155,7 +156,7 @@ jobs: run: tests/run-tests.py --print-failures float_clang: - runs-on: ubuntu-20.04 + runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Install packages @@ -172,6 +173,11 @@ jobs: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + # Python 3.12 is the default for ubuntu-24.04, but that has compatibility issues with settrace tests. + # Can remove this step when ubuntu-latest uses a more recent Python 3.x as the default. + with: + python-version: '3.11' - name: Build run: source tools/ci.sh && ci_unix_settrace_build - name: Run main test suite @@ -184,6 +190,11 @@ jobs: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + # Python 3.12 is the default for ubuntu-24.04, but that has compatibility issues with settrace tests. + # Can remove this step when ubuntu-latest uses a more recent Python 3.x as the default. + with: + python-version: '3.11' - name: Build run: source tools/ci.sh && ci_unix_settrace_stackless_build - name: Run main test suite @@ -193,7 +204,7 @@ jobs: run: tests/run-tests.py --print-failures macos: - runs-on: macos-11.0 + runs-on: macos-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v5 @@ -208,7 +219,8 @@ jobs: run: tests/run-tests.py --print-failures qemu_mips: - runs-on: ubuntu-latest + # ubuntu-22.04 is needed for older libffi. + runs-on: ubuntu-22.04 steps: - uses: actions/checkout@v4 - name: Install packages @@ -222,7 +234,8 @@ jobs: run: tests/run-tests.py --print-failures qemu_arm: - runs-on: ubuntu-latest + # ubuntu-22.04 is needed for older libffi. + runs-on: ubuntu-22.04 steps: - uses: actions/checkout@v4 - name: Install packages @@ -234,3 +247,38 @@ jobs: - name: Print failures if: failure() run: tests/run-tests.py --print-failures + + qemu_riscv64: + # ubuntu-22.04 is needed for older libffi. + runs-on: ubuntu-22.04 + steps: + - uses: actions/checkout@v4 + - name: Install packages + run: source tools/ci.sh && ci_unix_qemu_riscv64_setup + - name: Build + run: source tools/ci.sh && ci_unix_qemu_riscv64_build + - name: Run main test suite + run: source tools/ci.sh && ci_unix_qemu_riscv64_run_tests + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures + + sanitize_undefined: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Install packages + run: source tools/ci.sh && ci_unix_coverage_setup + - name: Build + run: source tools/ci.sh && ci_unix_sanitize_undefined_build + - name: Run main test suite + run: source tools/ci.sh && ci_unix_sanitize_undefined_run_tests + - name: Test merging .mpy files + run: source tools/ci.sh && ci_unix_coverage_run_mpy_merge_tests + - name: Build native mpy modules + run: source tools/ci.sh && ci_native_mpy_modules_build + - name: Test importing .mpy generated by mpy_ld.py + run: source tools/ci.sh && ci_unix_coverage_run_native_mpy_tests + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures diff --git a/.github/workflows/ports_windows.yml b/.github/workflows/ports_windows.yml index 4b1f4ea791e20..f33277d471d3c 100644 --- a/.github/workflows/ports_windows.yml +++ b/.github/workflows/ports_windows.yml @@ -18,7 +18,129 @@ concurrency: cancel-in-progress: true jobs: - build: + build-vs: + strategy: + fail-fast: false + matrix: + platform: [x86, x64] + configuration: [Debug, Release] + variant: [dev, standard] + visualstudio: ['2017', '2019', '2022'] + include: + - visualstudio: '2017' + vs_version: '[15, 16)' + - visualstudio: '2019' + vs_version: '[16, 17)' + - visualstudio: '2022' + vs_version: '[17, 18)' + # trim down the number of jobs in the matrix + exclude: + - variant: standard + configuration: Debug + - visualstudio: '2019' + configuration: Debug + runs-on: windows-latest + env: + CI_BUILD_CONFIGURATION: ${{ matrix.configuration }} + steps: + - name: Install Visual Studio 2017 + if: matrix.visualstudio == '2017' + run: | + choco install visualstudio2017buildtools + choco install visualstudio2017-workload-vctools + choco install windows-sdk-8.1 + - name: Install Visual Studio 2019 + if: matrix.visualstudio == '2019' + run: | + choco install visualstudio2019buildtools + choco install visualstudio2019-workload-vctools + choco install windows-sdk-8.1 + - uses: microsoft/setup-msbuild@v2 + with: + vs-version: ${{ matrix.vs_version }} + - uses: actions/checkout@v4 + - name: Build mpy-cross.exe + run: msbuild mpy-cross\mpy-cross.vcxproj -maxcpucount -property:Configuration=${{ matrix.configuration }} -property:Platform=${{ matrix.platform }} + - name: Update submodules + run: git submodule update --init lib/micropython-lib + - name: Build micropython.exe + run: msbuild ports\windows\micropython.vcxproj -maxcpucount -property:Configuration=${{ matrix.configuration }} -property:Platform=${{ matrix.platform }} -property:PyVariant=${{ matrix.variant }} + - name: Get micropython.exe path + id: get_path + run: | + $exePath="$(msbuild ports\windows\micropython.vcxproj -nologo -v:m -t:ShowTargetPath -property:Configuration=${{ matrix.configuration }} -property:Platform=${{ matrix.platform }} -property:PyVariant=${{ matrix.variant }})" + echo ("micropython=" + $exePath.Trim()) >> $env:GITHUB_OUTPUT + - name: Run tests + id: test + env: + MICROPY_MICROPYTHON: ${{ steps.get_path.outputs.micropython }} + working-directory: tests + run: python run-tests.py + - name: Print failures + if: failure() && steps.test.conclusion == 'failure' + working-directory: tests + run: python run-tests.py --print-failures + - name: Run mpy tests + id: test_mpy + env: + MICROPY_MICROPYTHON: ${{ steps.get_path.outputs.micropython }} + working-directory: tests + run: python run-tests.py --via-mpy -d basics float micropython + - name: Print mpy failures + if: failure() && steps.test_mpy.conclusion == 'failure' + working-directory: tests + run: python run-tests.py --print-failures + + build-mingw: + strategy: + fail-fast: false + matrix: + variant: [dev, standard] + sys: [mingw32, mingw64] + include: + - sys: mingw32 + env: i686 + - sys: mingw64 + env: x86_64 + runs-on: windows-latest + env: + CHERE_INVOKING: enabled_from_arguments + defaults: + run: + shell: msys2 {0} + steps: + - uses: actions/setup-python@v5 + # note: can go back to installing mingw-w64-${{ matrix.env }}-python after + # MSYS2 updates to Python >3.12 (due to settrace compatibility issue) + with: + python-version: '3.11' + - uses: msys2/setup-msys2@v2 + with: + msystem: ${{ matrix.sys }} + update: true + install: >- + make + mingw-w64-${{ matrix.env }}-gcc + pkg-config + git + diffutils + path-type: inherit # Remove when setup-python is removed + - uses: actions/checkout@v4 + - name: Build mpy-cross.exe + run: make -C mpy-cross -j2 + - name: Update submodules + run: make -C ports/windows VARIANT=${{ matrix.variant }} submodules + - name: Build micropython.exe + run: make -C ports/windows -j2 VARIANT=${{ matrix.variant }} + - name: Run tests + id: test + run: make -C ports/windows test_full VARIANT=${{ matrix.variant }} + - name: Print failures + if: failure() && steps.test.conclusion == 'failure' + working-directory: tests + run: python run-tests.py --print-failures + + cross-build-on-linux: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 diff --git a/.github/workflows/ports_zephyr.yml b/.github/workflows/ports_zephyr.yml index f6f328c9277e4..eb85af6a36154 100644 --- a/.github/workflows/ports_zephyr.yml +++ b/.github/workflows/ports_zephyr.yml @@ -20,10 +20,41 @@ jobs: build: runs-on: ubuntu-latest steps: + - uses: jlumbroso/free-disk-space@main + with: + # Only free up a few things so this step runs quickly. + android: false + dotnet: true + haskell: true + large-packages: false + docker-images: false + swap-storage: false - uses: actions/checkout@v4 + - id: versions + name: Read Zephyr version + run: source tools/ci.sh && echo "ZEPHYR=$ZEPHYR_VERSION" | tee "$GITHUB_OUTPUT" + - name: Cached Zephyr Workspace + id: cache_workspace + uses: actions/cache@v4 + with: + # note that the Zephyr CI docker image is 15GB. At time of writing + # GitHub caches are limited to 10GB total for a project. So we only + # cache the "workspace" + path: ./zephyrproject + key: zephyr-workspace-${{ steps.versions.outputs.ZEPHYR }} + - name: ccache + uses: hendrikmuhs/ccache-action@v1.2 + with: + key: zephyr - name: Install packages run: source tools/ci.sh && ci_zephyr_setup - name: Install Zephyr + if: steps.cache_workspace.outputs.cache-hit != 'true' run: source tools/ci.sh && ci_zephyr_install - name: Build run: source tools/ci.sh && ci_zephyr_build + - name: Run main test suite + run: source tools/ci.sh && ci_zephyr_run_tests + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures diff --git a/.github/workflows/ruff.yml b/.github/workflows/ruff.yml index 1159a1882d60e..4c4a2a3162ed6 100644 --- a/.github/workflows/ruff.yml +++ b/.github/workflows/ruff.yml @@ -1,11 +1,13 @@ -# https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python name: Python code lint and formatting with ruff + on: [push, pull_request] + jobs: ruff: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - - run: pip install --user ruff==0.1.3 + # ruff version should be kept in sync with .pre-commit-config.yaml & also micropython-lib + - run: pipx install ruff==0.11.6 - run: ruff check --output-format=github . - run: ruff format --diff . diff --git a/.gitignore b/.gitignore index 2d20cb18970e8..b5010dfd14db6 100644 --- a/.gitignore +++ b/.gitignore @@ -11,8 +11,9 @@ build/ build-*/ docs/genrst/ -# Test failure outputs +# Test failure outputs and intermediate artefacts tests/results/* +tests/ports/unix/ffi_lib.so # Python cache files __pycache__/ diff --git a/.gitmodules b/.gitmodules index 75bffdadddeb8..02849ec9bdd11 100644 --- a/.gitmodules +++ b/.gitmodules @@ -3,13 +3,13 @@ url = https://github.com/micropython/axtls.git [submodule "lib/libffi"] path = lib/libffi - url = https://github.com/atgreen/libffi + url = https://github.com/libffi/libffi [submodule "lib/lwip"] path = lib/lwip url = https://github.com/lwip-tcpip/lwip.git [submodule "lib/berkeley-db-1.xx"] path = lib/berkeley-db-1.xx - url = https://github.com/pfalcon/berkeley-db-1.xx + url = https://github.com/micropython/berkeley-db-1.xx [submodule "lib/stm32lib"] path = lib/stm32lib url = https://github.com/micropython/stm32lib @@ -59,3 +59,18 @@ [submodule "lib/protobuf-c"] path = lib/protobuf-c url = https://github.com/protobuf-c/protobuf-c.git +[submodule "lib/open-amp"] + path = lib/open-amp + url = https://github.com/OpenAMP/open-amp.git +[submodule "lib/libmetal"] + path = lib/libmetal + url = https://github.com/OpenAMP/libmetal.git +[submodule "lib/arduino-lib"] + path = lib/arduino-lib + url = https://github.com/arduino/arduino-lib-mpy.git +[submodule "lib/alif_ensemble-cmsis-dfp"] + path = lib/alif_ensemble-cmsis-dfp + url = https://github.com/alifsemi/alif_ensemble-cmsis-dfp.git +[submodule "lib/alif-security-toolkit"] + path = lib/alif-security-toolkit + url = https://github.com/micropython/alif-security-toolkit.git diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 3004bc5677214..ac9785bb59232 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -12,7 +12,16 @@ repos: verbose: true stages: [commit-msg] - repo: https://github.com/charliermarsh/ruff-pre-commit - rev: v0.1.3 + # Version should be kept in sync with .github/workflows/ruff.yml & also micropython-lib + rev: v0.11.6 hooks: - id: ruff - id: ruff-format + - repo: https://github.com/codespell-project/codespell + # Version should be kept in sync with .github/workflows/codespell.yml + rev: v2.4.1 + hooks: + - id: codespell + name: Spellcheck for changed files (codespell) + additional_dependencies: + - tomli diff --git a/CODECONVENTIONS.md b/CODECONVENTIONS.md index d6b0a5926bf4b..d3f71cb083deb 100644 --- a/CODECONVENTIONS.md +++ b/CODECONVENTIONS.md @@ -104,6 +104,22 @@ This command may work, please raise a new Issue if it doesn't: curl -L https://github.com/Homebrew/homebrew-core/raw/2b07d8192623365078a8b855a164ebcdf81494a6/Formula/uncrustify.rb > uncrustify.rb && brew install uncrustify.rb && rm uncrustify.rb ``` +Code spell checking +=================== + +Code spell checking is done using [codespell](https://github.com/codespell-project/codespell#codespell) +and runs in a GitHub action in CI. Codespell is configured via `pyproject.toml` +to avoid false positives. It is recommended run codespell before submitting a +PR. To simplify this, codespell is configured as a pre-commit hook and will be +installed if you run `pre-commit install` (see below). + +If you want to install and run codespell manually, you can do so by running: + +``` +$ pip install codespell tomli +$ codespell +``` + Automatic Pre-Commit Hooks ========================== @@ -190,14 +206,21 @@ adhere to the existing style and use `tools/codeformat.py` to check any changes. The main conventions, and things not enforceable via the auto-formatter, are described below. -White space: +As the MicroPython code base is over ten years old, not every source file +conforms fully to these conventions. If making small changes to existing code, +then it's usually acceptable to follow the existing code's style. New code or +major changes should follow the conventions described here. + +## White space + - Expand tabs to 4 spaces. - Don't leave trailing whitespace at the end of a line. - For control blocks (if, for, while), put 1 space between the keyword and the opening parenthesis. - Put 1 space after a comma, and 1 space around operators. -Braces: +## Braces + - Use braces for all blocks, even no-line and single-line pieces of code. - Put opening braces on the end of the line it belongs to, not on @@ -205,18 +228,43 @@ Braces: - For else-statements, put the else on the same line as the previous closing brace. -Header files: +## Header files + - Header files should be protected from multiple inclusion with #if directives. See an existing header for naming convention. -Names: +## Names + - Use underscore_case, not camelCase for all names. - Use CAPS_WITH_UNDERSCORE for enums and macros. - When defining a type use underscore_case and put '_t' after it. -Integer types: MicroPython runs on 16, 32, and 64 bit machines, so it's -important to use the correctly-sized (and signed) integer types. The -general guidelines are: +### Public names (declared in headers) + +- MicroPython-specific names (especially any declared in `py/` and `extmod/` + directories) should generally start with `mp_` or `MP_`. +- Functions and variables declared in a header should generally share a longer + common prefix. Usually the prefix matches the file name (i.e. items defined in + `py/obj.c` are declared in `py/obj.h` and should be prefixed `mp_obj_`). There + are exceptions, for example where one header file contains declarations + implemented in multiple source files for expediency. + +### Private names (specific to a single .c file) + +- For static functions and variables exposed to Python (i.e. a static C function + that is wrapped in `MP_DEFINE_CONST_FUN_...` and attached to a module), use + the file-level shared common prefix, i.e. name them as if the function or + variable was not static. +- Other static definitions in source files (i.e. functions or variables defined + in a .c file that are only used within that .c file) don't need any prefix + (specifically: no `s_` or `_` prefix, and generally avoid adding the + file-level common prefix). + +## Integer types + +MicroPython runs on 16, 32, and 64 bit machines, so it's important to use the +correctly-sized (and signed) integer types. The general guidelines are: + - For most cases use mp_int_t for signed and mp_uint_t for unsigned integer values. These are guaranteed to be machine-word sized and therefore big enough to hold the value from a MicroPython small-int @@ -225,11 +273,13 @@ general guidelines are: - You can use int/uint, but remember that they may be 16-bits wide. - If in doubt, use mp_int_t/mp_uint_t. -Comments: +## Comments + - Be concise and only write comments for things that are not obvious. - Use `// ` prefix, NOT `/* ... */`. No extra fluff. -Memory allocation: +## Memory allocation + - Use m_new, m_renew, m_del (and friends) to allocate and free heap memory. These macros are defined in py/misc.h. diff --git a/LICENSE b/LICENSE index 9bd457b42acea..929a2e97de7bf 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,6 @@ The MIT License (MIT) -Copyright (c) 2013-2023 Damien P. George +Copyright (c) 2013-2025 Damien P. George Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal @@ -48,16 +48,17 @@ used during the build process and is not part of the compiled source code. /cmsis (BSD-3-clause) /crypto-algorithms (NONE) /libhydrogen (ISC) + /libmetal (BSD-3-clause) /littlefs (BSD-3-clause) /lwip (BSD-3-clause) /mynewt-nimble (Apache-2.0) /nrfx (BSD-3-clause) /nxp_driver (BSD-3-Clause) /oofatfs (BSD-1-clause) + /open-amp (BSD-3-clause) /pico-sdk (BSD-3-clause) /re15 (BSD-3-clause) /stm32lib (BSD-3-clause) - /tinytest (BSD-3-clause) /tinyusb (MIT) /uzlib (Zlib) /wiznet5k (MIT) @@ -69,6 +70,9 @@ used during the build process and is not part of the compiled source code. /FreeRTOS (GPL-2.0 with FreeRTOS exception) /esp32 /ppp_set_auth.* (Apache-2.0) + /rp2 + /mutex_extra.c (BSD-3-clause) + /clocks_extra.c (BSD-3-clause) /stm32 /usbd*.c (MCD-ST Liberty SW License Agreement V2) /stm32_it.* (MIT + BSD-3-clause) diff --git a/README.md b/README.md index f5bc6d78f0a40..c78a2384604fe 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ Python 3.5 and some select features from later versions). The following core datatypes are provided: `str`(including basic Unicode support), `bytes`, `bytearray`, `tuple`, `list`, `dict`, `set`, `frozenset`, `array.array`, `collections.namedtuple`, classes and instances. Builtin modules include -`os`, `sys`, `time`, `re`, and `struct`, etc. Select ports have support for +`os`, `sys`, `time`, `re`, and `struct`, etc. Some ports have support for `_thread` module (multithreading), `socket` and `ssl` for networking, and `asyncio`. Note that only a subset of Python 3 functionality is implemented for the data types and modules. @@ -35,8 +35,8 @@ DAC, PWM, SPI, I2C, CAN, Bluetooth, and USB. Getting started --------------- -See the [online documentation](https://docs.micropython.org/) for API -references and information about using MicroPython and information about how +See the [online documentation](https://docs.micropython.org/) for the API +reference and information about using MicroPython and information about how it is implemented. We use [GitHub Discussions](https://github.com/micropython/micropython/discussions) @@ -108,13 +108,13 @@ track of the code size of the core runtime and VM. In addition, the following ports are provided in this repository: - [cc3200](ports/cc3200) -- Texas Instruments CC3200 (including PyCom WiPy). - - [esp32](ports/esp32) -- Espressif ESP32 SoC (including ESP32S2, ESP32S3, ESP32C3). + - [esp32](ports/esp32) -- Espressif ESP32 SoC (including ESP32S2, ESP32S3, ESP32C3, ESP32C6). - [esp8266](ports/esp8266) -- Espressif ESP8266 SoC. - [mimxrt](ports/mimxrt) -- NXP m.iMX RT (including Teensy 4.x). - [nrf](ports/nrf) -- Nordic Semiconductor nRF51 and nRF52. - [pic16bit](ports/pic16bit) -- Microchip PIC 16-bit. - [powerpc](ports/powerpc) -- IBM PowerPC (including Microwatt) - - [qemu-arm](ports/qemu-arm) -- QEMU-based emulated target, for testing) + - [qemu](ports/qemu) -- QEMU-based emulated target (for testing) - [renesas-ra](ports/renesas-ra) -- Renesas RA family. - [rp2](ports/rp2) -- Raspberry Pi RP2040 (including Pico and Pico W). - [samd](ports/samd) -- Microchip (formerly Atmel) SAMD21 and SAMD51. diff --git a/docs/conf.py b/docs/conf.py index 728e3c4285d82..eb61487582024 100755 --- a/docs/conf.py +++ b/docs/conf.py @@ -36,6 +36,9 @@ "is_release": micropy_version != "latest", } +# Authors used in various parts of the documentation. +micropy_authors = "MicroPython authors and contributors" + # -- General configuration ------------------------------------------------ @@ -68,7 +71,7 @@ # General information about the project. project = "MicroPython" -copyright = "- The MicroPython Documentation is Copyright © 2014-2023, Damien P. George, Paul Sokolovsky, and contributors" +copyright = "- The MicroPython Documentation is Copyright © 2014-2025, " + micropy_authors # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the @@ -244,7 +247,7 @@ master_doc, "MicroPython.tex", "MicroPython Documentation", - "Damien P. George, Paul Sokolovsky, and contributors", + micropy_authors, "manual", ), ] @@ -281,7 +284,7 @@ "index", "micropython", "MicroPython Documentation", - ["Damien P. George, Paul Sokolovsky, and contributors"], + [micropy_authors], 1, ), ] @@ -300,7 +303,7 @@ master_doc, "MicroPython", "MicroPython Documentation", - "Damien P. George, Paul Sokolovsky, and contributors", + micropy_authors, "MicroPython", "One line description of project.", "Miscellaneous", diff --git a/docs/develop/cmodules.rst b/docs/develop/cmodules.rst index 75dbc953c06f8..c5aa919b7645a 100644 --- a/docs/develop/cmodules.rst +++ b/docs/develop/cmodules.rst @@ -59,7 +59,7 @@ A MicroPython user C module is a directory with the following files: SRC_USERMOD_LIB_C += $(EXAMPLE_MOD_DIR)/utils/algorithm.c Similarly, use ``SRC_USERMOD_CXX`` and ``SRC_USERMOD_LIB_CXX`` for C++ - source files. + source files. If you want to include assembly files use ``SRC_USERMOD_LIB_ASM``. If you have custom compiler options (like ``-I`` to add directories to search for header files), these should be added to ``CFLAGS_USERMOD`` for C code diff --git a/docs/develop/compiler.rst b/docs/develop/compiler.rst index cac92585ff4ee..0c25ad3a01ef1 100644 --- a/docs/develop/compiler.rst +++ b/docs/develop/compiler.rst @@ -98,7 +98,7 @@ Then also edit ``py/lexer.c`` to add the new keyword literal text: .. code-block:: c :emphasize-lines: 12 - STATIC const char *const tok_kw[] = { + static const char *const tok_kw[] = { ... "or", "pass", @@ -157,7 +157,7 @@ The most relevant method you should know about is this: mp_compile_to_raw_code(parse_tree, source_file, is_repl, &cm); // Create and return a function object that executes the outer module. - return mp_make_function_from_raw_code(cm.rc, cm.context, NULL); + return mp_make_function_from_proto_fun(cm.rc, cm.context, NULL); } The compiler compiles the code in four passes: scope, stack size, code size and emit. @@ -301,7 +301,7 @@ code statement: .. code-block:: c - STATIC void emit_native_unary_op(emit_t *emit, mp_unary_op_t op) { + static void emit_native_unary_op(emit_t *emit, mp_unary_op_t op) { vtype_kind_t vtype; emit_pre_pop_reg(emit, &vtype, REG_ARG_2); if (vtype == VTYPE_PYOBJ) { diff --git a/docs/develop/gettingstarted.rst b/docs/develop/gettingstarted.rst index a17a320671f52..fed632ea1ac15 100644 --- a/docs/develop/gettingstarted.rst +++ b/docs/develop/gettingstarted.rst @@ -112,7 +112,7 @@ Check that you have Python available on your system: .. code-block:: bash $ python3 - Python 3.5.0 (default, Jul 17 2020, 14:04:10) + Python 3.5.0 (default, Jul 17 2020, 14:04:10) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> @@ -278,7 +278,7 @@ To run a selection of tests on a board/device connected over USB use: .. code-block:: bash $ cd tests - $ ./run-tests.py --target minimal --device /dev/ttyACM0 + $ ./run-tests.py -t /dev/ttyACM0 See also :ref:`writingtests`. diff --git a/docs/develop/library.rst b/docs/develop/library.rst index c2a86ea1699bc..830211d81c862 100644 --- a/docs/develop/library.rst +++ b/docs/develop/library.rst @@ -48,16 +48,16 @@ hypothetical new module ``subsystem`` in the file ``modsubsystem.c``: #if MICROPY_PY_SUBSYSTEM // info() - STATIC mp_obj_t py_subsystem_info(void) { + static mp_obj_t py_subsystem_info(void) { return MP_OBJ_NEW_SMALL_INT(42); } MP_DEFINE_CONST_FUN_OBJ_0(subsystem_info_obj, py_subsystem_info); - STATIC const mp_rom_map_elem_t mp_module_subsystem_globals_table[] = { + static const mp_rom_map_elem_t mp_module_subsystem_globals_table[] = { { MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_subsystem) }, { MP_ROM_QSTR(MP_QSTR_info), MP_ROM_PTR(&subsystem_info_obj) }, }; - STATIC MP_DEFINE_CONST_DICT(mp_module_subsystem_globals, mp_module_subsystem_globals_table); + static MP_DEFINE_CONST_DICT(mp_module_subsystem_globals, mp_module_subsystem_globals_table); const mp_obj_module_t mp_module_subsystem = { .base = { &mp_type_module }, diff --git a/docs/develop/natmod.rst b/docs/develop/natmod.rst index 6d15f867bcfe8..2ccd83288565c 100644 --- a/docs/develop/natmod.rst +++ b/docs/develop/natmod.rst @@ -39,7 +39,8 @@ options for the ``ARCH`` variable, see below): * ``armv7emsp`` (ARM Thumb 2, single precision float, eg Cortex-M4F, Cortex-M7) * ``armv7emdp`` (ARM Thumb 2, double precision float, eg Cortex-M7) * ``xtensa`` (non-windowed, eg ESP8266) -* ``xtensawin`` (windowed with window size 8, eg ESP32) +* ``xtensawin`` (windowed with window size 8, eg ESP32, ESP32S3) +* ``rv32imc`` (RISC-V 32 bits with compressed instructions, eg ESP32C3, ESP32C6) When compiling and linking the native .mpy file the architecture must be chosen and the corresponding file can only be imported on that architecture. For more @@ -69,11 +70,25 @@ The known limitations are: So, if your C code has writable data, make sure the data is defined globally, without an initialiser, and only written to within functions. +The native module is not automatically linked against the standard static libraries +like ``libm.a`` and ``libgcc.a``, which can lead to ``undefined symbol`` errors. +You can link the runtime libraries by setting ``LINK_RUNTIME = 1`` +in your Makefile. Custom static libraries can also be linked by adding +``MPY_LD_FLAGS += -l path/to/library.a``. Note that these are linked into +the native module and will not be shared with other modules or the system. + Linker limitation: the native module is not linked against the symbol table of the full MicroPython firmware. Rather, it is linked against an explicit table of exported symbols found in ``mp_fun_table`` (in ``py/nativeglue.h``), that is fixed at firmware build time. It is thus not possible to simply call some arbitrary HAL/OS/RTOS/system -function, for example. +function, for example, unless that resides at a fixed address. In that case, the path +of a linkerscript containing a series of symbol names and their fixed address can be +passed to ``mpy_ld.py`` via the ``--externs`` command line argument. That way symbols +appearing in the linkerscript will take precedence over what is provided from object +files, but at the moment the object files' implementation will still reside in the +final MPY file. The linkerscript parser is limited in its capabilities, and is +currently used only for parsing the ESP8266 port ROM symbols list (see +``ports/esp8266/boards/eagle.rom.addr.v6.ld``). New symbols can be added to the end of the table and the firmware rebuilt. The symbols also need to be added to ``tools/mpy_ld.py``'s ``fun_table`` dict in the @@ -128,7 +143,7 @@ The file ``factorial.c`` contains: #include "py/dynruntime.h" // Helper function to compute factorial - STATIC mp_int_t factorial_helper(mp_int_t x) { + static mp_int_t factorial_helper(mp_int_t x) { if (x == 0) { return 1; } @@ -136,7 +151,7 @@ The file ``factorial.c`` contains: } // This is the function which will be called from Python, as factorial(x) - STATIC mp_obj_t factorial(mp_obj_t x_obj) { + static mp_obj_t factorial(mp_obj_t x_obj) { // Extract the integer from the MicroPython input object mp_int_t x = mp_obj_get_int(x_obj); // Calculate the factorial @@ -145,7 +160,7 @@ The file ``factorial.c`` contains: return mp_obj_new_int(result); } // Define a Python reference to the function above - STATIC MP_DEFINE_CONST_FUN_OBJ_1(factorial_obj, factorial); + static MP_DEFINE_CONST_FUN_OBJ_1(factorial_obj, factorial); // This is the entry point and is called when the module is imported mp_obj_t mpy_init(mp_obj_fun_bc_t *self, size_t n_args, size_t n_kw, mp_obj_t *args) { @@ -172,7 +187,7 @@ The file ``Makefile`` contains: # Source files (.c or .py) SRC = factorial.c - # Architecture to build for (x86, x64, armv6m, armv7m, xtensa, xtensawin) + # Architecture to build for (x86, x64, armv6m, armv7m, xtensa, xtensawin, rv32imc) ARCH = x64 # Include to get the rules for compiling and linking the module diff --git a/docs/develop/optimizations.rst b/docs/develop/optimizations.rst index 7f2c8cbe7282c..533dd64116f03 100644 --- a/docs/develop/optimizations.rst +++ b/docs/develop/optimizations.rst @@ -33,7 +33,7 @@ Variables MicroPython processes local and global variables differently. Global variables are stored and looked up from a global dictionary that is allocated on the heap (note that each module has its own separate dict, so separate namespace). -Local variables on the other hand are are stored on the Python value stack, which may +Local variables on the other hand are stored on the Python value stack, which may live on the C stack or on the heap. They are accessed directly by their offset within the Python stack, which is more efficient than a global lookup in a dict. diff --git a/docs/develop/porting.rst b/docs/develop/porting.rst index 63919b97a6ccd..99725e1000bef 100644 --- a/docs/develop/porting.rst +++ b/docs/develop/porting.rst @@ -83,7 +83,7 @@ The basic MicroPython firmware is implemented in the main port file, e.g ``main. } // There is no filesystem so opening a file raises an exception. - mp_lexer_t *mp_lexer_new_from_file(const char *filename) { + mp_lexer_t *mp_lexer_new_from_file(qstr filename) { mp_raise_OSError(MP_ENOENT); } @@ -262,17 +262,17 @@ To add a custom module like ``myport``, first add the module definition in a fil #include "py/runtime.h" - STATIC mp_obj_t myport_info(void) { + static mp_obj_t myport_info(void) { mp_printf(&mp_plat_print, "info about my port\n"); return mp_const_none; } - STATIC MP_DEFINE_CONST_FUN_OBJ_0(myport_info_obj, myport_info); + static MP_DEFINE_CONST_FUN_OBJ_0(myport_info_obj, myport_info); - STATIC const mp_rom_map_elem_t myport_module_globals_table[] = { + static const mp_rom_map_elem_t myport_module_globals_table[] = { { MP_OBJ_NEW_QSTR(MP_QSTR___name__), MP_OBJ_NEW_QSTR(MP_QSTR_myport) }, { MP_ROM_QSTR(MP_QSTR_info), MP_ROM_PTR(&myport_info_obj) }, }; - STATIC MP_DEFINE_CONST_DICT(myport_module_globals, myport_module_globals_table); + static MP_DEFINE_CONST_DICT(myport_module_globals, myport_module_globals_table); const mp_obj_module_t myport_module = { .base = { &mp_type_module }, diff --git a/docs/develop/writingtests.rst b/docs/develop/writingtests.rst index 9bb5178f55f22..a7d033f17e9b9 100644 --- a/docs/develop/writingtests.rst +++ b/docs/develop/writingtests.rst @@ -60,7 +60,7 @@ Then to run on a board: .. code-block:: bash - $ ./run-tests.py --target minimal --device /dev/ttyACM0 + $ ./run-tests.py -t /dev/ttyACM0 And to run only a certain set of tests (eg a directory): diff --git a/docs/differences/modules_preamble.txt b/docs/differences/modules_preamble.txt new file mode 100644 index 0000000000000..1958f0084d2db --- /dev/null +++ b/docs/differences/modules_preamble.txt @@ -0,0 +1,33 @@ +.. Preamble section inserted into generated output + +Positional-only Parameters +-------------------------- + +To save code size, many functions that accept keyword arguments in CPython only accept positional arguments in MicroPython. + +MicroPython marks positional-only parameters in the same way as CPython, by inserting a ``/`` to mark the end of the positional parameters. Any function whose signature ends in ``/`` takes *only* positional arguments. For more details, see `PEP 570 `_. + +Example +~~~~~~~ + +For example, in CPython 3.4 this is the signature of the constructor ``socket.socket``:: + + socket.socket(family=AF_INET, type=SOCK_STREAM, proto=0, fileno=None) + +However, the signature documented in :func:`MicroPython` is:: + + socket(af=AF_INET, type=SOCK_STREAM, proto=IPPROTO_TCP, /) + +The ``/`` at the end of the parameters indicates that they are all positional-only in MicroPython. The following code works in CPython but not in most MicroPython ports:: + + import socket + s = socket.socket(type=socket.SOCK_DGRAM) + +MicroPython will raise an exception:: + + TypeError: function doesn't take keyword arguments + +The following code will work in both CPython and MicroPython:: + + import socket + s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) diff --git a/docs/differences/python_37.rst b/docs/differences/python_37.rst index 86d1b6e81f96d..2b9a089217f6d 100644 --- a/docs/differences/python_37.rst +++ b/docs/differences/python_37.rst @@ -105,4 +105,4 @@ Changes to built-in modules: .. rubric:: Notes -.. [#ftimenanosec] Only :func:`time.time_ns` is implemented. +.. [#ftimenanosec] Only :func:`time.time_ns` is implemented. diff --git a/docs/esp32/quickref.rst b/docs/esp32/quickref.rst index 08df25b0554d0..c394414a76f51 100644 --- a/docs/esp32/quickref.rst +++ b/docs/esp32/quickref.rst @@ -18,6 +18,9 @@ working with this board it may be useful to get an overview of the microcontroll general.rst tutorial/index.rst +Note that there are several varieties of ESP32 -- ESP32, ESP32C3, ESP32C6, ESP32S2, ESP32S3 -- +supported by MicroPython, with some differences in functionality between them. + Installing MicroPython ---------------------- @@ -58,47 +61,53 @@ The :mod:`esp32` module:: import esp32 esp32.raw_temperature() # read the internal temperature of the MCU, in Fahrenheit - esp32.ULP() # access to the Ultra-Low-Power Co-processor + esp32.ULP() # access to the Ultra-Low-Power Co-processor, not on ESP32C3/C6 Note that the temperature sensor in the ESP32 will typically read higher than ambient due to the IC getting warm while it runs. This effect can be minimised by reading the temperature sensor immediately after waking up from sleep. +ESP32C3, ESP32C6, ESP32S2, and ESP32S3 also have an internal temperature sensor available. +It is implemented a bit differently to the ESP32 and returns the temperature in +Celsius:: + + esp32.mcu_temperature() # read the internal temperature of the MCU, in Celsius + Networking ---------- WLAN ^^^^ -The :mod:`network` module:: +The :class:`network.WLAN` class in the :mod:`network` module:: import network - wlan = network.WLAN(network.STA_IF) # create station interface - wlan.active(True) # activate the interface - wlan.scan() # scan for access points - wlan.isconnected() # check if the station is connected to an AP + wlan = network.WLAN() # create station interface (the default, see below for an access point interface) + wlan.active(True) # activate the interface + wlan.scan() # scan for access points + wlan.isconnected() # check if the station is connected to an AP wlan.connect('ssid', 'key') # connect to an AP - wlan.config('mac') # get the interface's MAC address - wlan.ifconfig() # get the interface's IP/netmask/gw/DNS addresses + wlan.config('mac') # get the interface's MAC address + wlan.ipconfig('addr4') # get the interface's IPv4 addresses - ap = network.WLAN(network.AP_IF) # create access-point interface - ap.config(ssid='ESP-AP') # set the SSID of the access point - ap.config(max_clients=10) # set how many clients can connect to the network - ap.active(True) # activate the interface + ap = network.WLAN(network.WLAN.IF_AP) # create access-point interface + ap.config(ssid='ESP-AP') # set the SSID of the access point + ap.config(max_clients=10) # set how many clients can connect to the network + ap.active(True) # activate the interface A useful function for connecting to your local WiFi network is:: def do_connect(): - import network - wlan = network.WLAN(network.STA_IF) + import machine, network + wlan = network.WLAN() wlan.active(True) if not wlan.isconnected(): print('connecting to network...') wlan.connect('ssid', 'key') while not wlan.isconnected(): - pass - print('network config:', wlan.ifconfig()) + machine.idle() + print('network config:', wlan.ipconfig('addr4')) Once the network is established the :mod:`socket ` module can be used to create and use TCP/UDP sockets as usual, and the ``requests`` module for @@ -112,31 +121,56 @@ calling ``wlan.config(reconnects=n)``, where n are the number of desired reconne attempts (0 means it won't retry, -1 will restore the default behaviour of trying to reconnect forever). +.. _esp32_network_lan: + LAN ^^^ -To use the wired interfaces one has to specify the pins and mode :: +Built-in MAC (original ESP32) +""""""""""""""""""""""""""""" + +The original ESP32 SoC has a built-in Ethernet MAC. Using this MAC requires an +external Ethernet PHY to be wired to the chip's EMAC pins. Most of the EMAC pin +assignments are fixed, consult the ESP32 datasheet for details. + +If the PHY is connected, the internal Ethernet MAC can be configured via +the :class:`network.LAN` constructor:: import network lan = network.LAN(mdc=PIN_MDC, ...) # Set the pin and mode configuration lan.active(True) # activate the interface - lan.ifconfig() # get the interface's IP/netmask/gw/DNS addresses + lan.ipconfig('addr4') # get the interface's IPv4 addresses + + +Required keyword arguments for the constructor: + +- ``mdc`` and ``mdio`` - :class:`machine.Pin` objects (or integers) specifying + the MDC and MDIO pins. +- ``phy_type`` - Select the PHY device type. Supported devices are + ``PHY_GENERIC``, + ``PHY_LAN8710``, ``PHY_LAN8720``, ``PHY_IP101``, ``PHY_RTL8201``, + ``PHY_DP83848``, ``PHY_KSZ8041`` and ``PHY_KSZ8081``. These values are all + constants defined in the ``network`` module. +- ``phy_addr`` - The address number of the PHY device. Must be an integer in the + range 0x00 to 0x1f, inclusive. Common values are ``0`` and ``1``. +All of the above keyword arguments must be present to configure the interface. -The keyword arguments for the constructor defining the PHY type and interface are: +Optional keyword arguments: -- mdc=pin-object # set the mdc and mdio pins. -- mdio=pin-object -- power=pin-object # set the pin which switches the power of the PHY device. -- phy_type= # Select the PHY device type. Supported devices are PHY_LAN8710, - PHY_LAN8720, PH_IP101, PHY_RTL8201, PHY_DP83848 and PHY_KSZ8041 -- phy_addr=number # The address number of the PHY device. -- ref_clk_mode=mode # Defines, whether the ref_clk at the ESP32 is an input - or output. Suitable values are Pin.IN and Pin.OUT. -- ref_clk=pin-object # defines the Pin used for ref_clk. +- ``reset`` - :class:`machine.Pin` object (or integer) specifying the PHY reset pin. +- ``power`` - :class:`machine.Pin` object (or integer) specifying a pin which + switches the power of the PHY device. +- ``ref_clk`` - :class:`machine.Pin` object (or integer) specifying the pin used + for the EMAC ``ref_clk`` signal. If not specified, the board default is used + (typically GPIO 0, but may be different if a particular board has Ethernet.) +- ``ref_clk_mode`` - Defines whether the EMAC ``ref_clk`` pin of the ESP32 + should be an input or an output. Suitable values are ``machine.Pin.IN`` and + ``machine.Pin.OUT``. If not specified, the board default is used + (typically input, but may be different if a particular board has Ethernet.) -These are working configurations for LAN interfaces of popular boards:: +These are working configurations for LAN interfaces of some popular ESP32 boards:: # Olimex ESP32-GATEWAY: power controlled by Pin(5) # Olimex ESP32 PoE and ESP32-PoE ISO: power controlled by Pin(12) @@ -161,6 +195,66 @@ These are working configurations for LAN interfaces of popular boards:: lan = network.LAN(id=0, mdc=Pin(23), mdio=Pin(18), power=Pin(5), phy_type=network.PHY_IP101, phy_addr=1) + +.. _esp32_spi_ethernet: + +SPI Ethernet Interface +"""""""""""""""""""""" + +All ESP32 SoCs support external SPI Ethernet interface chips. These are Ethernet +interfaces that connect via a SPI bus, rather than an Ethernet RMII interface. + +.. note:: The only exception is the ESP32 ``d2wd`` variant, where this feature is disabled + to save code size. + +SPI Ethernet uses the same :class:`network.LAN` constructor, with a different +set of keyword arguments:: + + import machine, network + + spi = machine.SPI(1, sck=SCK_PIN, mosi=MOSI_PIN, miso=MISO_PIN) + lan = network.LAN(spi=spi, cs=CS_PIN, ...) # Set the pin and mode configuration + lan.active(True) # activate the interface + lan.ipconfig('addr4') # get the interface's IPv4 addresses + +Required keyword arguments for the constructor: + +- ``spi`` - Should be a :class:`machine.SPI` object configured for this + connection. Note that any clock speed configured on the SPI object is ignored, + the SPI Ethernet clock speed is configured at compile time. +- ``cs`` - :class:`machine.Pin` object (or integer) specifying the CS pin + connected to the interface. +- ``int`` - :class:`machine.Pin` object (or integer) specifying the INT pin + connected to the interface. +- ``phy_type`` - Select the SPI Ethernet interface type. Supported devices are + ``PHY_KSZ8851SNL``, ``PHY_DM9051``, ``PHY_W5500``. These values are all + constants defined in the ``network`` module. +- ``phy_addr`` - The address number of the PHY device. Must be an integer in the + range 0x00 to 0x1f, inclusive. This is usually ``0`` for SPI Ethernet devices. + +All of the above keyword arguments must be present to configure the interface. + +Optional keyword arguments for the constructor: + +- ``reset`` - :class:`machine.Pin` object (or integer) specifying the SPI Ethernet + interface reset pin. +- ``power`` - :class:`machine.Pin` object (or integer) specifying a pin which + switches the power of the SPI Ethernet interface. + +Here is a sample configuration for a WIZNet W5500 chip connected to pins on +an ESP32-S3 development board:: + + import machine, network + from machine import Pin, SPI + + spi = SPI(1, sck=Pin(12), mosi=Pin(13), miso=Pin(14)) + lan = network.LAN(spi=spi, phy_type=network.PHY_W5500, phy_addr=0, + cs=Pin(10), int=Pin(11)) + +.. note:: WIZnet W5500 Ethernet is also supported on some other MicroPython + ports, but using a :ref:`different software interface + `. + Delay and timing ---------------- @@ -177,8 +271,10 @@ Use the :mod:`time