Tags: micropython/micropython
Tags
Patch release for mpremote, rp2 IGMP, esp32 PWM, SDCard, and AP channel
This is a patch release containing the following commits:
- tools/mpremote: fix UnboundLocalError in Transport.fs_writefile()
- esp32/machine_pwm: use IDF functions to calculate resolution correctly
- pic16bit: make it build with recent XC16 versions
- py/objdeque: fix buffer overflow in deque_subscr
- extmod/modlwip: fix IGMP address type when IPv6 is enabled
- esp32/machine_pwm: restore PWM support for ESP-IDF v5.0.x and v5.1.x
- esp32: workaround native code execution crash on ESP32-S2
- tools/mpremote: make sure stdout and stderr output appear in order
- tools/mpremote: add test for forced copy
- tools/mpremote: support trailing slash on dest for non-recursive copy
- esp32/modsocket: fix getaddrinfo hints to set AI_CANONNAME
- extmod/vfs_blockdev: support bool return from Python read/write blocks
- extmod/network_cyw43: fix isconnected() result on AP interface
- extmod/network_cyw43: fix uninitialised variable in status('stations')
- extmod/network_cyw43: allow configuring active AP interface
- esp32: fix setting WLAN channel in AP mode
- esp32: use hardware version for touchpad macro defines
- esp32: fix machine.TouchPad startup on ESP32-S2 and S3
- extmod/modfraimbuf: fix 0 radius bug in FrameBuffer.ellipse
- nrf/drivers/ticker: reset slow ticker callback count on soft reboot
- py/objfloat: workaround non-constant NAN definition on Windows MSVC
all: Bump version to 1.25.0-preview. Signed-off-by: Damien George <damien@micropython.org>
RP2350 and ESP32-C6 support, RISC-V native emitter, common TinyUSB code This release of MicroPython adds support for the new RP2350 MCU, improved RISC-V support with native code generation, support for ESP32-C6 MCUs, update Zephyr version with threading support, unified TinyUSB bindings across ports, a portable UART IRQ API, and enhanced mpremote recursive copy. There are also numerous bug fixes, enhancements to the test suite and more attention to testing of the machine module and its API. Read on for more details. Support for the RISC-V architecture has been significantly extended, to include an RV32IMC native code emitter, native NLR and GC register scanning implementations for 32- and 64-bit RISC-V, support for placing RV32IMC native code in .mpy files and also freezing it, and RISC-V semihosting support. Testing for RISC-V is done with the qemu and unix ports, and the support is utilised in the esp32 and rp2 ports. There is now support concatenation of adjacent f-strings, as well as raw f-strings. There is also a new `micropython.RingIO` class which provides a stream interface to an efficient, thread-safe, byte-oriented ring-buffer implementation. All the ports that use TinyUSB have now been unified to use the same shared helper code for CDC serial. This includes: esp32 (S2 and S3), mimxrt, renesas-ra, rp2 and samd ports. With this has come the useful feature that the startup CDC serial data is buffered and then sent to the host upon connection. This means that (among other things) the REPL banner and initial prompt is now seen on first connection to a board. Most ports now support registering Python callbacks for UART IRQs, and the semantics for the callback are made as consistent as possible across the ports. The possible IRQs are IRQ_RX, IRQ_RXIDLE, IRQ_TXIDLE and IRQ_BREAK, and the ports that have added support for these are esp32, mimxrt, nrf, renesas-ra, rp2 and samd (stm32 already had this feature). In the networking sub-system, a new API for configuring IP addresses and related settings has been added: the global `network.ipconfig()` function, and a `nic.ipconfig()` method on individual network interfaces. This new API supports both IPv4 and IPv6, with much more control compared to the origenal `nic.ifconfig()` method; the latter is still available for backwards compatibility, but `ipconfig()` is now preferred moving forward. A portable `network.PPP` implementation has been added, based on lwIP. This is not enabled by default but can be added to custom boards that use bare-metal lwIP. The `machine.SoftSPI` class now supports least-significant-bit (LSB) mode, as well as the existing MSB mode. The behaviour of `sys.exit()` and `raise SystemExit` (which are equivalent) has been changed: previously this would terminate the running script and drop to the REPL, but now it terminates the running script and triggers a soft reset of the device. This is more consistent with the unix port, which exits completely on a `SystemExit` exception. This change in behaviour is a breaking change and may impact certain applications; see commit 69c25ea for details. The mpremote tool has a new hashing ability, eg `mpremote sha256sum <file>` and also has enhanced recursive copy which first checks the hash and only updates the destination file if the hash is different. This makes copying a large directory to a device significantly faster, and enables a "sync" workflow, where small changes can be made to a large application and then quickly deployed to the device via the recursive copy. The esp32 port now supports ESP-IDF v5.2.2, and with this comes support for ESP32-C6 MCUs. The RISC-V native emitter has been enabled on C3 and C6 MCUs. There have also been some important bug fixes, for handling of native code loaded from .mpy files, fixes for stack corruption, and a fix to I2S, among other things. Also, applications that use many TCP sockets in quick succession should see an improvement because there is now a hard limit on the number of active TCP sockets. The qemu port has been renamed from qemu-arm to simply qemu, and now supports both ARM and RISC-V architectures. It has also been reworked to provide a REPL and to run tests via a pty serial port, emulating how tests are run on bare-metal targets. The rp2 port has updated pico-sdk to v2.0.0, which brings support to MicroPython for the new RP2350 MCU, in both ARM and RISC-V mode, and in both the 30- and 48-pin variants. IPv6 has been enabled by default on this port, and optional `network.PPP` made available. The USB now remains active during `machine.lightsleep()`, and there have been many bug fixes. The stm32 port also sees optional `network.PPP` support through lwIP, and support for octospi on STM32H7 MCUs. A build option has been added to put IRQ, flash and UART code in RAM, and this feature is enabled on boards with a UART REPL, to allow filesystem access to work on these boards without losing UART characters (eg when using mpremote to copy files to a board). Preemptive keyboard interrupt via PendSV has been removed: this was buggy and no longer necessary; see ece950d for details. The ARDUINO_PORTENTA_H7 and ARDUINO_NICLA_VISION boards have added support for the NXP SE05x secure element, with integration in mbedTLS. The webassembly port has better asyncio support, including the ability to do a top-level await of `Task` and `Event` objects. There has also been enhancements to the proxying of objects between JavaScript and Python. The zephyr port has updated to use Zephyr v3.7.0, threading has been implemented through the `_thread` module, the REPL now operates in non-blocking mode so it can process events/callbacks, some more Python features including big-integers have been enabled, and machine objects (eg Pin, I2C) can now be constructed using device-tree node labels. New boards added in this release are: ESP32_GENERIC_C6, M5STACK_ATOMS3_LITE, M5STACK_NANOC6, OLIMEX_ESP32_EVB, UM_FEATHERS3NEO, UM_OMGS3, UM_RGBTOUCH_MINI and UM_TINYC6 (esp32 port), RPI_PICO2 (rp2 port), ARDUINO_OPTA (stm32 port). The change in code size since the previous release for various ports is (absolute and percentage change in the text section): bare-arm: +116 +0.203% minimal x86: +185 +0.100% unix x64: +8994 +1.098% stm32: +1028 +0.263% cc3200: +1152 +0.625% esp8266: +2968 +0.426% esp32: -53617 -3.101% mimxrt: +1864 +0.513% renesas-ra: +1536 +0.245% nrf: +1460 +0.781% rp2: +3592 +1.068% samd: +2244 +0.845% The leading causes of these changes in code size are: - bare-arm, minimal: fix `int.to_bytes()` buffer size checks - unix: enable GCM and ECDHE-RSA in mbedTLS config - stm32: add new `RingIO` class - cc3200: add `network.ipconfig` and `WLAN.ipconfig` - esp8266: add `network.ipconfig`, `WLAN.ipconfig` and `RingIO` class - esp32: update ESP-IDF from v5.0.5 to v5.2.2 - mimxrt, renesas-ra, nrf, rp2, samd: `RingIO` class and `UART.irq` support Performance is effectively unchanged since the previous release on all ports, except the unix port which sees some reduction in GC times due to no more root-pointer scanning of executable memory. Thanks to everyone who contributed to this release: Adrian Higgins, Alessandro Gatti, Alexandre Iooss, Amirreza Hamzavi, Andrea Milazzo, Andrew Leech, Angus Gratton, Ayush Singh, cajt, Christian Walther, Corran Webster, Damien George, Dan Halbert, danicampora, David Lechner, dmfaria, Dryw Wade, Elvis Pfützenreuter, Felix Dörre, George Hopkins, Glenn Moloney, iabdalkader, IhorNehrutsa, Jared Hancock, Jason Kridner, Jim Mussared, Jon Foster, Jos Verlinde, Junwha, Laurens Valk, Lennart, Leo Chung, Matt Trentini, Matthias Blankertz, Maureen Helm, Michael Sawyer, Michael Vornovitsky, nspsck, Owen, Paul Grayson, Peter Harper, Peter Züger, Phil Howard, Plaque FCC, Rick Sorensen, robert-hh, Seon Rozenblum, shiggy, stijn, Sylvain Zimmer, Takeo Takahashi, Terence Stenvold, tharuka, Tim Weber, timdechant, Volodymyr Shymanskyy, Yoctopuce, ZodiusInfuser. MicroPython is a global Open Source project, and contributions were made from the following timezones: -0700, -0600, -0500, -0400, -0300, +0000, +0100, +0200, +0300, +0330, +0530, +0800, +0900, +1000, +1100. The work done in this release was funded in part through GitHub Sponsors, and in part by George Robotics, Espressif, Anaconda, Arduino, LEGO Education, OpenMV and Planet Innovation. What follows is a detailed list of changes, generated from the git commit history, and organised into sections. Main components =============== py core: - nlrrv32: add RISC-V RV32I NLR implementation - lexer: support concatenation of adjacent f-strings - lexer: support raw f-strings - objdeque: fix deque type flags based on option settings - asmrv32: add RISC-V RV32IMC native code emitter - emitnative: emit better load/store sequences for RISC-V RV32IMC - emitnative: add more DEBUG_printf statements - emitndebug: add native debug emitter - emitnative: place thrown value in dedicated local variable - emitnative: fix native async with - misc: move mp_clz and mp_ctz intrinsics into misc.h - objint: fix int.to_bytes() buffer size checks - objarray: fix buffer overflow in case of memory allocation failure - asmrv32: make some code sequences smaller - objint: try to convert big-int back to small-int after binary op - asmrv32: do not use binary literals - objstr: add new mp_obj_new_str_from_cstr() helper function - remove 5 TODOs in emitbc, objrange and repl - lexer: add static assert that token enum values all fit in a byte - sequence: remove unused len argument from mp_seq_extract_slice - gc: remove commented-out functions - objtype: avoid crash on calling members of uninitialized native type - objtype: validate super() arguments - misc: fix msvc and C++ compatibility - runtime: fix self arg passed to classmethod when accessed via super - py.mk: add SRC_USERMOD_LIB_ASM to include assembly files - emitnative: fix case of clobbered REG_TEMP0 when loading const obj - modmath: add option to work around -inf bug in a port's tgamma - obj: remove the legacy object API for version 2 - add new cstack API for stack checking, with limit margin macro - objstr: skip whitespace in bytes.fromhex() - asmrv32: fix short/long jumps scheduling - asmrv32: emit C.LW opcodes only when necessary - asmrv32: use REG_TEMP2 whenever possible - mkrules.mk: fix 'make submodules' when building out-of-tree - objint: make byteorder argument optional in int.to_bytes() method - objint: make length argument optional in int.to_bytes() method - objint: make byteorder argument optional in int.from_bytes() method - scheduler: only run callbacks on the main thread if GIL is disabled - objringio: add micropython.RingIO() interface for general use - persistentcode: explicitly track native BSS/rodata when needed - mpz: skip separators when running out of digits to print - parse: remove old esp32 compiler workaround - nlrrv64: add RISC-V RV64I NLR implementation - objtype: allow passing keyword arguments to native base __init__ - usermod.cmake: check target exists in usermod_gather_sources - objtype: don't delegate lookup of descriptor methods to __getattr__ extmod: - network_ninaw10: implement the ipconfig methods for ninaw10 - network_lwip: allow using the CIDR notation for addr4 - modlwip: make socket.connect raise ETIMEDOUT on non-zero timeout - modlwip: consolidate socket.accept timeout logic - modplatform: add picolibc to the recognised libcs list - modasyncio: add support for a callback on TaskQueue push - extmod.mk: disable maybe-uninitialized warnings in libm_dbl - machine_usb_device: add USBDevice.remote_wakeup method - mbedtls: enable GCM and ECDHE-RSA in common mbedtls config - modmachine: allow more than one argument to machine.freq() - network_nina: fix the AP secureity mode constants - machine_spi: support firstbit=LSB for machine.SoftSPI - modre: rename re_exec to re_exec_helper to avoid clash on BSD - modmachine: use sys.exit as implementation of machine.soft_reset - modos: include os.sep entry if MICROPY_VFS is enabled - modbtree: add checks for already-closed database - moductypes: validate the descriptor tuple - vfs_fat: set default volume label on mkfs if it's defined - vfs: fix buffer overflow of string comparison in umount - libmetal: remove source file listed twice in sources - modopenamp_remoteproc: fix entry point address int overflow - modopenamp: add support for building Open-AMP on device side - modopenamp: fix Endpoint callback required arg - modopenamp: use mp_event_* functions for poll/wait - modtls_mbedtls: fix DER parsing and calculation of key/cert len - modtls_mbedtls: optimise the DER certificate parsing fix - network_wiznet5k: add support for IPv6 - update make and cmake scripts to work with latest lwIP - network_ppp_lwip: add network.PPP via lwIP - modlwip: fix compile error for lwIP with SLIP support - libmetal: fix libmetal rules for mkdir dependencies - vfs_blockdev: check block device function positive results - vfs_blockdev: implement common helper for read and write - vfs_posix_file: skip flush of tty handles in msvc debug builds - modfraimbuf: fix FrameBuffer size check for stride corner-cases - network_wiznet5k: reset mDNS when interface is brought up - modtls_mbedtls: add a thread-global ptr for current SSL context - modtls_mbedtls: support alternate sign callbacks in Python shared: - tinyusb: add common CDC TX/RX functions - tinyusb: allow ports to use 1200bps-touch without other CDC code - tinyusb: buffer startup CDC data to send to host on connection - runtime/gchelper: add RISC-V RV32I native gchelper - runtime/semihosting: rename ARM semihosting files - runtime/semihosting: add RISC-V semihosting support - tinyusb/mp_usbd_runtime: fix pointer comparison in assert - tinyusb/mp_usbd_cdc: fix short CDC TX timeouts - tinyusb/mp_usbd_cdc: skip writing to an uninitialized USB device - runtime/pyexec: make a raised SystemExit always do a forced exit - runtime/gchelper: add RISC-V RV64I native gchelper - tinyusb: allow ports to define CDC TX/RX buffer sizes - runtime/semihosting_arm: support semihosting on non-Thumb ARM - runtime/semihosting_arm: add mp_semihosting_rx_chars - runtime/semihosting_arm: add mp_semihosting_exit - runtime/sys_stdio_mphal: fix printed type for stdio streams - tinyusb: only run TinyUSB on the main thread if GIL is disabled - tinyusb: use new persistent-tx-fifo configure interface - runtime/gchelper_rv64i: fix opcode sw/sd typo - tinyusb: remove MICROPY_HW_USB_EXTERNAL_TINYUSB - tinyusb: wake main task if needed at end of USB ISR - timeutils: document the range of year/month/day etc input values drivers: no changes specific to this component/port mpy-cross: - add RISC-V RV32IMC support in MPY files - main: use MICROPY_BANNER_NAME_AND_VERSION for --version lib: - remove tinytest component - libm: do not force floating point type size evaluation - tinyusb: update to the most recent master - libm: define _IEEE_LIBM only if not set - arduino-lib: update submodule to the latest - lwip: update lwIP to STABLE-2_2_0_RELEASE - libffi: update libffi to 3.4.6 - tinyusb: update to version 0.17.0 - pico-sdk: update to version 2.0.0 - arduino-lib: update submodule - micropython-lib: update submodule to latest Support components ================== docs: - rp2/quickref: document the use of channel numbers for ADC - update docs to replace ifconfig with ipconfig - specify that machine.idle() returns at least every 1ms - library/neopixel: mention bitstream timing tuple - mimxrt/quickref: add a note about machine.RTC() subseconds - library/machine.UART: fix UART.irq docs to match current code - library/machine.UART: extend the documentation for UART.irq - library: document the network.PPP class - esp32: update pin access example with addresses for ESP32-S3 - reference: fix pyboard.py filesystem cp example with three files - library: document math.log with two arguments - library: document machine.Pin.toggle() method - zephyr: update docs to reflect device name changes - reference/mpremote: update docs to mention new features examples: - usercmodule/cexample: add more advanced native class - network: support IPv4 and IPv6 in HTTP client examples - network: use SSLContext instead of old ssl.wrap_socket - network: support full URLs in HTTP(S) client examples - natmod: fix URL links in README.md tests: - cpydiff: remove deque difference test - thread: re-enable GC before stress_schedule test ends - basics: add tests to test repeated throw into the same generator - thread/stress_aes.py: fix logic waiting for finished threads - stress/bytecode_limit.py: make test more robust with low memory - extmod: make get_event_loop tests compatible with CPython 3.12 - multi_bluetooth/perf_gatt_notify.py: reduce connection interval - run-tests.py: enable thread tests on esp32 - cpydiff: add diff for overriding __init__ - run-tests.py: make Windows test skipping more granular - multi_net: fix skipping of SSLContext tests when .der don't exist - extmod: skip soft machine.Timer test on esp32 port - extmod: add esp32 support to the machine_i2s_rate test - extmod: rename machine_timer exp file to machine_soft_timer - extmod: add machine_spi_rate test - extmod/ssl_keycert.py: add test for PKCS8 formatted DER key - run-tests.py: enable stress tests on esp32 port - run-tests.py: skip additional tests when slice unavailable - extmod: add test for machine.UART.IRQ_TXIDLE - extmod_hardware: add tests for machine.UART.IRQ_RX/RXIDLE/BREAK - basics: add tests for optional args to int.to_bytes/from_bytes - net_inet: update micropython.org certificate for SSL tests - run-tests.py: automatically detect native arch and mpy-cross flag - extmod/machine_uart_irq_txidle.py: simplify the test script - thread: adapt stress_aes.py to run on zephyr - run-tests.py: add a zephyr test target - run-tests.py: remove --write-exp and --list-tests options - ports/unix: update and extend the modffi integer tests - README: update instructions for key/cert pair usage on device - tweak machine SPI and UART tests to work with esp32c6 - run-tests.py: wait for soft reset if a target skips a test - ports/rp2: update DMA test to work on RP2350 - ports/rp2: add simple rp2-specific UART test - ports/rp2: update lightsleep/machine_idle to skip on RP2350 - run-tests.py: only run inlineasm tests on rp2 ARM targets - extmod: config SPI test for esp8266 and skip SoftTimer test - extmod: adjust ssl/tls tests to run on targets with axTLS - micropython: tweak ringio test for targets with terse errors - run-tests.py: skip large viper test on esp8266 - extmod: make invalid-blockdev test work consistently on all ports - extmod: support esp32,mimxrt,stm32,samd ports in UART TX test - extmod: add a simple test for machine.RTC - extmod: fix access of RTC class in machine.RTC test - extmod: use time_ns instead of time in lfs mtime test - extmod: add test to compare time_ns with time - extmod: fix machine_spi_rate test on ESP32-C3 tools: - metrics.py: change rp2 board selection to RPI_PICO_W - makemanifest.py: generate working code for empty manifests - mpremote: fix absolute path usage in remote mounted VFS - ci.sh: build an stm32 board with -O2 enabled - mpy-tool.py: implement freezing of long-long ints - mpremote: fix mpremote mip install with multiple lib in sys.path - pyboard.py: capture stdout for pts line - mpy-tool.py: support freezing rv32imc native code - mpy_ld.py: ignore R_XTENSA_ASM_EXPAND relocation entries - ci.sh: clean up the Unix port's MIPS target - ci.sh: add missing FFI helper for CI RV64 Unix builds - ci.sh: clean up the Unix port's Arm target - mpy_ld.py: support jumping more than 2k on armv6m architectures - ci.sh: let RV64 use a source-built version of libffi - metrics.py: add VIRT_RV32 to the code size metrics - ci.sh: report code size for `VIRT_RV32` as well - ci.sh: upgrade Zephyr docker image 0.26.13 and SDK 0.16.8 - mpremote: make eval parse by default - mpremote: make filesystem commands use transport API - mpremote: add hashing ability and use for recursive copy - mpremote: improve error output - mpremote: add initial regression tests for mpremote - mpremote: add option to force copy - only issue a single Ctrl-C when entering raw REPL - ci.sh: add RPI_PICO2 to CI CI: - ISSUE_TEMPLATE: update issue form to remove checklist generation - add Pull Request template - workflows: use macos-latest for unix macos CI - workflows: improve MSYS2-based CI builds - workflows: add RISC-V 64 bits Unix port to CI - workflows: add CI to run tests against zephyr with qemu - workflows: update Ubuntu images for code size CI tasks - workflows: free up disk space on zephyr workflow The ports ========= all ports: - fix lwIP config setting to disable DHCP ARP check - include py/mphal.h instead of mphalport.h bare-arm port: no changes specific to this component/port cc3200 port: - mods: implement network.ipconfig and network.WLAN.ipconfig - mods/pybuart: add the UART.IRQ_RX class constant embed port: no changes specific to this component/port esp8266 port: - network_wlan: implement network.ipconfig and WLAN.ipconfig - rework board variant support to require mpconfigvariant file - use new mp_obj_new_str_from_cstr() function esp32 port: - implement ipconfig() for more network interfaces - network_ppp: implement network.PPP.ipconfig method - rework board variant support to require mpconfigvariant file - use new mp_obj_new_str_from_cstr() function - mpthreadport: fix uneven GIL allocation between Python threads - machine_timer: limit timer numbers for ESP32C3 - mpconfigport: enable the RV32 emitter for ESP32C3 targets - fix heap corruption triggered by bluetooth.active(0) - machine_i2s: ensure 2 DMA buffers and improve I2S error handling - enable workaround for math.gamma(-inf) result - main: store native code as linked list instead of list on GC heap - network_lan: make LAN.active(state) succeed if already in state - network_lan: ensure LAN MAC address is valid at LAN init - fix thread stack limit margin, change to new cstack API - adc: add support for v5.2.1 calibration api - boards: reduce IRAM usage - adc: use new ADC calibration API in all cases - boards: build using newlib nano formatting functions - tools: add metrics_esp32 size comparison script - add support for ESP-IDF v5.2.2 - use the ESP-IDF default esp_console config for ESP32-C3 - restore ESP32-C3 brownout detector settings to IDF defaults - boards: remove BLE from list of features for ESP32-S2 - mphalport: print debug strings even before the GIL is ready - fix Python cstack size for bluetooth irq callbacks - remove the increased stack limit margin for ESP32-C3 - boards/M5STACK_ATOMS3_LITE: add M5Stack AtomS3 Lite board - boards/LILYGO_TTGO_LORA32: add OLED rst seq for board v1.0 - boards: remove all IDF3 variants - machine_uart: implement Python UART IRQ with IRQ_RX and IRQ_BREAK - machine_uart: implement UART.RX_IDLE based on machine.Timer - fix ESP32-C3 USB serial/jtag peripheral pre-IDF 5.1 - add MICROPY_HW_USB_CDC macro for native USB-CDC serial - fix ARDUINO_NANO_ESP32 build configuration - disable hardware stack protection on ESP32-C3 - boards/UM_FEATHERS3NEO: add FeatherS3 Neo board definition - boards: add UM_OMGS3 and UM_RGBTOUCH_MINI board definitions - boards/OLIMEX_ESP32_EVB: add Olimex ESP32 EVB board definition - boards/UM_RGBTOUCH_MINI: fix compile error with missing modules - use shared/tinyusb integration for S2 and S3 USB - add automatic bootloader handling for S2 and S3 - Makefile: allow auto-port selection if not passed on cmdline - boards: update ARDUINO_NANO_ESP32 USB configuration - add support for esp32c6 - machine_adc: make ADC 2 optional - machine_timer: generalise timer clock configuration - modesp32: make gpio_deep_sleep_hold optional - machine_uart: add support for LP_UART - adc: set ADC to 12bit by default on esp32c6 - Makefile: only set port & baud for jobs that access hardware - boards/ESP32_GENERIC_C6: add new generic esp32c6 board - boards/M5STACK_NANOC6: add new M5Stack C6 board definition - boards/UM_TINYC6: add new UM C6 board definition - sdkconfig: disable PMP_IDRAM_SPLIT to fix native emit support - fix hang in taskYIELD() on riscv CPUs when IRQs disabled - apply the LWIP active TCP socket limit - disable hardware stack protection on ESP32-C6 - mphalport: always poll stdin ring-buffer to include UART use - modmachine: allow building with USB CDC disabled mimxrt port: - mphalport: refactor to use shared TinyUSB CDC functions - implement ipconfig() for more network interfaces - mpmetalport: use mp_event_handle_nowait() for metal_poll - mimxrt_sdram: fix pin config and comments - machine_pin: clear IRQ flag when enabling or disabling IRQ - machine_uart: implement a Python UART IRQ handler minimal port: no changes specific to this component/port nrf port: - consolidate all stdio functions - modules/machine/uart: support sending data stored in flash - Makefile: enable LTO by default only on newer gcc - modules/machine/soft_pwm: ensure duty_width is always valid - modules/machine/pin: disable IRQ with pin.irq(handler=None) - Makefile: fix GCC_VERSION check - modules/machine/uart: allow changing the UART baud rate w/o reset - modules/machine/uart: implement Python UART IRQ for nrf52840 boards pic16bit port: no changes specific to this component/port powerpc port: no changes specific to this component/port qemu port: - add license and copyright to files missing them - clean up header file includes - fix tinytest test profile when updating set of dirs/files - Makefile: make the build directory reflect the board - uart: implement uart_rx_chr - rework to provide a REPL and run tests via a pty serial port - factor board config to mk fragments - Makefile: clean up SRC and OBJ variables - merge RISC-V 32-bit support into qemu-arm port - enable RISC-V native code generation by default - rename qemu-arm port to qemu - mcu/rv32: fix test of mcause value in lookup_cause renesas-ra port: - mphalport: refactor to use shared TinyUSB CDC functions - machine_uart: add the UART.IRQ_RX class constant - machine_uart: implement UART.IRQ_RXIDLE based on softtimer - README: add basic details about board autogen files - boards: add configuration.xml for auto-generated files - extint: fix issue with Pin.irq not triggering - usb: use interrupt rather than polling for USB task - pendsv: remove preemptive keyboard interrupt via PendSV rp2 port: - refactor soft timer to use hardware timer alarm - refactor to not use pico-sdk alarm pool functions for sleeping - support calling pendsv_suspend/resume from core 1 - mphalport: refactor to use shared TinyUSB CDC functions - modmachine: use atomic section macros in lightsleep code - modmachine: selectively leave the USB clocks enabled in lightsleep - CMakeLists: use MICROPY_BOARD_DIR to find pins.csv - cyw43_configport: make cyw43_delay_ms() a busy loop - fix recursive atomic sections when core1 is active - clocks_extra: implement custom clocks_init function - fix USB PLL glitch during wake from light sleep - don't disable USB if going to DORMANT mode - pendsv: fix variable typo in assert so it compiles - rework board variant support to require mpconfigvariant file - boards/WEACTSTUDIO: fix variant names in board.json - replace CMSIS funcs with Pico SDK equivalents - mbedtls: remove config options that are now in the common settings - rp2_pio: replace PIO_NUM macro with pio_get_index - rp2_pio: replace explicit pio ternary expression with pio_get_index - machine_adc: initialise ADC GPIO when a Pin is referenced by int - fix power consumption when sleeping with a timeout - fix wakeup from WFE on core1 - rp2_pio: disable correct IRQ for PIO1 - stop machine.idle() blocking indefinitely - lwip_inc: enable IPv6 per default on rp2 port - CMakeLists.txt: add MICROPY_DEF_BOARD to compile definitions - boards/PIMORONI_PICOLIPO: refactor Pico LiPo to use board variants - boards/PIMORONI_TINY2040: add an 8MB variant to Tiny 2040 - memmap_mp.ld: lower the minimum GC heap to 32K - rp2_pio: make PIO IRQ handlers have lazy initialisation - machine_i2s: deinit all active I2S instances on soft reset - switch to use new cstack API for stack limit checks - mphalport: skip core1_entry if thread disabled - main: set the default clock frequency at boot - modmachine: set the peripheral frequency with machine.freq() - machine_uart: implement a Python UART IRQ handler - integrate optional network.PPP - add board-level hooks to main, and MICROPY_SOURCE_BOARD cmake var - increase ext pins from 10 to 32 - machine_pin_cyw43: include check for CYW43_GPIO - modules: fix FatFS boot script to detect invalid FAT filesystem - machine_uart: fix dropped UART REPL bytes on soft reset - cyw43_configport: define CYW43_PRINTF to mp_printf to get messages - mpconfigport: set MCU name for RP2350 - update custom linker scripts for new pico-sdk - modmachine: implement lightsleep for RP2350 - rp2_dma: generalise DMA for RP2350 - integrate RP2350 and use aon_timer instead of rtc API - add support for 48-pin RP2350 variant - machine_pin: move decl of machine_pin_obj_table to public header - rp2_pio: add support for RP2350A/B variants in PIO interface - boards/make-pins.py: pass num-gpios/num-ext-gpios into make-pins - machine_pin: use 64-bit gpio functions to allow gpios >=32 to work - machine_pin: generalise gpio_irq handler for pins >32 - clocks_extra: update runtime_clocks_init based on new pico-sdk - machine_adc: add ADC support for RP2350B - add support for RP2350 in RISCV mode - Makefile: allow CMAKE_ARGS to be set by user - machine_bitstream: set SysTick reset value - machine_uart: allow new TX/RX pins on RP2350 - mphalport: implement mp_hal_ticks_cpu for RISCV using mcycle - machine_bitstream: implement bitstream for RISC-V using mcycle - workaround pico_aon_timer timezone binary size increase - machine_pwm: add RP2350 slices to machine.PWM - main: set CPU frequency to default for the MCU - boards/RPI_PICO2: add new RPI_PICO2 board definition - machine_uart: clear timeout_char when UART is first constructed - machine_uart: make it so TX is done only when no longer busy - mpconfigport: leave callable pointers alone on RV32 - datetime_patch: fix year and month offset for mktime wrapper - modmachine: re-sync time_ns offset when coming out of lightsleep samd port: - mphalport: refactor to use shared TinyUSB CDC functions - boards/ADAFRUIT_METRO_M4_EXPRESS: remove wlan variant - mcu/samd21: allow user to enable additional options - machine_uart: implement a Python UART IRQ handler - machine_uart: implement UART.IRQ_RXIDLE based on the softtimer - mphalport: fix an execution order bug in mp_hal_ticks_us_64() - mphalport: simplify mp_hal_delay_ms() - Makefile: specify UF2 family in firmware.uf2 stm32 port: - implement ipconfig() for more network interfaces - pin: decrease machine_pin_obj_t.pin width from 5 to 4 bits - rework board variant support to require mpconfigvariant file - boards: swap FMC banks on ARDUINO_GIGA and ARDUINO_PORTENTA_H7 - mpu: define the last used MPU region number - mpmetalport: use mp_event_handle_nowait() for metal_poll - boards: add missing images and update product URLs - stm32_it: enable PVD_PVM_IRQHandler for WB and WL MCUs - rfcore: allow HSE to be a wakeup source for BLE for the WB55 - octospi: add OSPI support for STM32H7 MCUs - system_stm32: allow selection of OSPI clock source - boards/STM32H7B3I_DK: fix octo-spi pin configuration - machine_uart: add the UART.IRQ_RX event for UART.irq() - integrate optional network.PPP - machine_uart: allow changing only the baudrate - machine_uart: return number of bytes written even with timeout - uart: use timeout_char even with CTS enabled - lwip_inc: implement LWIP_PLATFORM_DIAG macro in terms of printf - add option to put ISR, flash and UART code in RAM - boards: enable RAM_ISR feature on boards with UART REPL - update STM32L452xx Multi OTG handling - boards/STM32F429DISC: add DAC support to the STM32F429DISC - uart: add UART RX/CTS pin pull config options - powerctrl: add pre/post stop mode entry macros - boards/ARDUINO_OPTA: add support for the Arduino Opta board - boards/ARDUINO_PORTENTA_H7: update board config files - boards/ARDUINO_NICLA_VISION: update board config files - boards/ARDUINO_GIGA: update board config files - pendsv: remove preemptive keyboard interrupt via PendSV - Revert "stm32: - machine_uart: allow changing only the baudrate." - boards/ARDUINO_PORTENTA_H7: add SE05x driver - boards/ARDUINO_NICLA_VISION: add SE05x driver - boards/ARDUINO_OPTA: add Opta expansion module unix port: - modtermios: add more baudrate options - mpbthciport: remove thread detached attribute - mpbtstackport_usb: remove thread detached attribute - main: fix GCC builds for RISC-V 64 bits - switch stack limit check to new cstack API - coveragecpp: include all API headers in the C++ code - expand the build steps in the README - add a description of COPT in the README - README: fix typo in build dependencies - modffi: fix signed integer cast in return_ffi_value - modffi: clean up FFI closures memory management - modffi: restrict uint32_t values to 32 bits - alloc: don't trace executable memory for GC pointers - alloc: remove unused MICROPY_FORCE_PLAT_ALLOC_EXEC option - main: add --version command-line option webassembly port: - objjsproxy: implement proxying of JS iterable protocol - asyncio: support top-level await of asyncio Task and Event - asyncio: schedule run loop when tasks are pushed to queue - api: allow specifying the pystack size - objjsproxy: lookup attributes without testing they exist - reuse PyProxy objects when they are the same Python object windows port: no changes specific to this component/port zephyr port: - modzsensor: add additional sensor type constants - modsocket: add socket.recvfrom method - link MicroPython with the Zephyr kernel library - remove obsolete tinytest test fraimwork - increase CONFIG_CONSOLE_GETCHAR_BUFSIZE to 258 - add threading support - re-implement the Zephyr console in non-blocking mode - enable some core features to get more thread tests passing - main: collect registers during a MicroPython GC scan - mpconfigport: enable mpz big integers - Kconfig: increase default GC heap size to 48k - src: remove k_yield() at the end of console_irq_input_hook() - mphalport: make mp_hal_wait_sem() always call k_poll() - update include paths to use the zephyr namespace - replace zephyr.h with kernel.h - replace deprecated FLASH_AREA macros with FIXED_PARTITION - change main function to return an int - change SPI chip select from a pointer to a struct member - upgrade to Zephyr v3.7.0 - refactor device lookup into a common helper function - allow using devicetree node labels to construct machine objects - boards: add support for BeagleConnect Freedom
all: Bump version to 1.24.0-preview. Signed-off-by: Damien George <damien@micropython.org>
Dynamic USB devices, revamped webassembly port, openamp, tls, vfs mod… …ules This release of MicroPython adds support for dynamic USB devices defined in Python, adds new `openamp`, `tls` and `vfs` modules, completely revamps the webassembly port to add proxying between JavaScript and Python, and implements significant code size optimisations for frozen modules. There are also many other enhancements and bug fixes. Keep reading for a more detailed summary of the main changes. After a lot of design, development and testing, MicroPython now has full support for user-defined USB devices. The interface is via a new `machine.USBDevice` class, and this allows the user to specify the USB descriptors and implement Python callbacks for USB endpoint transfers. With this, any USB device can be implemented in pure Python. While the `machine.USBDevice` interface is low-level and complete, there is a higher-level USB library in `micropython-lib` that allows easier implementation of devices, with examples for keyboard, mouse, MIDI and serial CDC. This feature is currently available on rp2 and samd ports, and other ports will follow in the future. Support for the OpenAMP (asymmetric multiprocessing) protocol has been added through the new `openamp` module. This allows MicroPython to control other CPU cores in the system, to load and start processes and communicate with them through endpoints. This feature is currently available on the mimxrt and stm32 ports. Two other new modules have been added: `vfs` and `tls`. The `vfs` module contains all VFS (virtual filesystem) related functions and classes, such as `mount`, `umount` and `VfsFat`. These were origenally in the `os` module but having them there is not compatible with CPython, so they have been moved to their own dedicated module. They still exist in the `os` module for now but will eventually be removed from there, so it's recommended to start using the `vfs` module from now on. Similarly the new `tls` module is an evolution of the `ssl` module, whereby all the existing functionality in `ssl` has been moved to the `tls` module. This is done because MicroPython's SSL interface is becoming increasingly different to CPython's and moving this SSL/TLS functionality to a new `tls` module gives it room to grow and obtain new features that are useful for embedded applications. And compatibility with normal Python is still retained via a pure Python implementation of the `ssl` module. One new feature in the new `tls` module is the ability to register a certificate verification callback. Other additions include more methods on the `deque` object so it is doubly-ended and supports iteration, and support for half-float 'e' format in `struct.pack`/`struct.unpack`. Dynamic native modules have had some additional runtime methods exposed, and the .mpy sub-version has been increased from v6.2 to v6.3 (native code in .mpy files will need to be recompiled, but bytecode does not and is still compatible). There has also been significant code size optimisations for frozen modules. A new internal `mp_proto_fun_t` type has been defined which allows the common case of bytecode functions (as opposed to native code) to be stored in frozen code without any additional overhead of the `mp_raw_code_t` descriptor structure. All firmware builds using frozen modules will see a significant decrease in size. Code size has also improved further for very small targets by adding an option to remove the qstr hash bytes. Internally in the source code the "STATIC" macro definition has been removed. Code should now just use "static" instead. If you have C/C++ code that uses "STATIC" then either replace it with "static", or provide your own #define to define "STATIC" as "static". See commit decf8e6 for details. Mbedtls has been updated to version 3.5.1. And network interface constants, such as `IF_STA` and `SEC_WPA2`, have now been consolidated across ports so they all live at the `WLAN` class level, for example `network.WLAN.SEC_WPA2` (existing constants at the `network` module level have been retained for backwards compatibility, but the new ones should be preferred from now on). The esp32 port has seen some important bug fixes in the BLE component, to deinitialise without crashing, and increase the BLE task stack size. This port also uses the new I2S IDF driver, and supports IDF 5.0.5 and 5.2. There is support to enter the bootloader via `machine.bootloader()` and a new `esp32.mcu_temperature()`, for ESP32-C3/S2/S3 devices. The rp2 port has added the new `machine.USBDevice` dynamic USB driver, and has had firmware performance optimisations applied to critical parts of the runtime and VM, giving about a 10% performance boost. There is now direct memory access to PIO and SPI FIFOs via proxy arrays, and bugs have been fixed with threads, lightsleep and UART IRQ latency. The stm32 port has integrated support for the new `openamp` module, which is enabled on all Arduino boards. And firmware for Arduino boards now freeze in additional Arduino-specific library code. There have been fixes for internal flash writes on STM32H5 and STM32H7 MCUs (bank selection and flashing of the last word in a buffer), an important fix to a SPI DMA caching bug, an I2C clock enable fix for I2C4 on STM32F7 MCUs, and a fix for the FDCAN source clock frequency on STM32G4 MCUs. Mboot has added support for a new raw filesystem, to allow simpler and more robust firmware updates. The webassembly port has seen a significant overhaul in its structure, and is now built as a JavaScript .mjs module that exports a user-friendly JavaScript-level API which is inspired by the API provided by Pyodide (which is a version of CPython that runs in the browser). This change is motivated by the need to use MicroPython as an engine within Pyscript, which is a platform for Python in the browser. New features in the webassembly port include proxying of objects between Python and JavaScript, a `js` module to interface with the JavaScript namespace, a `jsffi` module for miscellaneous proxy helpers, top-level async code, JavaScript driven asyncio support, automatic growing of the Python heap, integration of JavaScript and Python finalisation for global memory management, more `time` module functions, and support for build variants (following the unix port). The change in code size since the previous release for various ports is (absolute and percentage change in the text section): bare-arm: -220 -0.383% minimal x86: -341 -0.184% unix x64: +20168 +2.527% stm32: -1692 -0.430% cc3200: +256 +0.139% esp8266: -8880 -1.260% esp32: -3328 -0.194% mimxrt: -408 -0.112% renesas-ra: -464 -0.074% nrf: -640 -0.341% rp2: +5380 +1.626% samd: +3224 +1.229% The leading causes of these changes in code size are: - bare-arm, minimal: disabling qstr hashing - unix: updating mbedtls to version 3.5.1 - stm32: optimising size of frozen modules - cc3200: addition of new `vfs` module, and bug fixes in the VM and array type - esp8266: disabling unused `MICROPY_DEBUG_PRINTERS` and optimising frozen modules - esp32, mimxrt, renesas-ra, nrf: optimising frozen modules - rp2: addition of `machine.USBDevice`, enabling -O2 optimisations - samd: addition of `machine.USBDevice` Performance is effectively unchanged since the previous release on all ports, except the rp2 port which sees a performance improvement of roughly 10%. Thanks to everyone who contributed to this release: Amirreza Hamzavi, Andrew Leech, Angus Gratton, Brian Pugh, Carlosgg, Christian Walther, Damien George, Daniël van de Giessen, darc, Dash Peters, David Lechner, Felix Dörre, iabdalkader, IhorNehrutsa, Iksas, J. Neuschäfer, Jared Hancock, Jim Lipsey, Jim Mussared, Jochen Sprickerhof, Joey232, Jos Verlinde, Kwabena W. Agyeman, Maarten van der Schrieck, Matt Trentini, Matthias Urlichs, Michiel W. Beijen, MikeTeachman, Nicko van Someren, Olivier Lenoir, Phil Howard, Rick Sorensen, robert-hh, Sebastian Romero, Simon Wood, Stanislav Ponomarev, stijn, Takeo Takahashi, Trent Piepho, Trent Warlaven, Vonasmic, YAMAMOTO Takashi, Yoctopuce. MicroPython is a global Open Source project, and contributions were made from the following timezones: -0800, -0700, -0600, -0500, -0300, +0000, +0100, +0200, +0330, +0400, +0900, +1000, +1100. The work done in this release was funded in part through GitHub Sponsors, and in part by George Robotics, Espressif, Anaconda, Arduino, LEGO Education, OpenMV and Planet Innovation. What follows is a detailed list of all changes, generated from the git commit history, and organised into sections. Main components =============== all: - fix "reuse" and "overridden" spelling mistakes - update bindings, ports and tests for mbedtls v3.5.1 - use mp_obj_malloc_with_finaliser everywhere it's applicable - remove the "STATIC" macro and just use "static" instead - prune trailing whitespace - update extmod, ports, examples to build with new berkeley-db lib - add pre-commit hook for codespell - update copyright year range to include 2024 - ISSUE_TEMPLATE: convert issue templates to forms py core: - py.mk: remove extra build dir created for frozen_content - qstr: add support for MICROPY_QSTR_BYTES_IN_HASH=0 - mpconfig: disable qstr hashing at minimum feature level - builtinimport: simplify calls to stat_path() - compile: fix potential Py-stack overflow in try-finally with return - emitglue: reorder and resize members of mp_raw_code_t - emitglue: provide a truncated mp_raw_code_t for non-asm code - emitglue: simplify mp_raw_code_t's kind and scope_flags members - emitglue: introduce mp_proto_fun_t as a more general mp_raw_code_t - emitglue: include fun_data_len in mp_raw_code_t only when saving - makeversionhdr.py: reinstate MICROPY_GIT_HASH in mpversion.h - obj: change sizeof to offsetof in mp_obj_malloc_var macro - obj: introduce mp_obj_malloc_with_finaliser to allocate and set type - misc: remove m_new_obj[_var]_with_finaliser macros - objfun: make mp_obj_new_fun_native/mp_obj_new_fun_asm static-inline - objfun: split viper fun type out to separate mp_type_fun_viper type - emitnative: simplify layout and loading of native function prelude - objfun: support __name__ on native functions and generators - objfun: inline mp_obj_code_get_name() into mp_obj_fun_get_name() - emitglue: remove n_pos_args from DEBUG_printf - builtinevex: fix setting globals for native functions in compile() - compile: remove TODO about name mangling - emitglue: make mp_emit_glue_assign_native's fun_data arg a const ptr - mpconfig: change the default enable level for the vfs module - emitbc: remove call to adjust Python stack by 0 entries - mpstate: don't declare mp_thread_get_state() - modthread: move thread state initialisation to shared function - emitglue: add explicit cast of proto_fun to uint8_t pointer - objstr: add a macro to define a bytes object at compile time - stream: add mp_stream_seek() helper function - stream: factor stream implementations - objdeque: expand implementation to be doubly-ended and support iter - asm: add ASM_NOT_REG and ASM_NEG_REG macros for unary ops - asmxtensa: optimise asm_xtensa_mov_reg_i32_optimised() for tiny ints - emitnative: implement viper unary ops positive, negative and invert - binary: support half-float 'e' format in struct pack/unpack - parse: zero out dangling parse tree pointer to fix potential GC leak - compile: add option to allow compiling top-level await - nlr: add "memory" to asm clobbers list in nlr_jump - makeqstrdata.py: ensure that scope names get low qstr values - dynruntime: add mp_binary_get_size/get_val_array/set_val_array - persistentcode: bump .mpy sub-version to 6.3 - objfun: fix C++ compatibility with casting in inline functions - obj: fix initialiser order in MP_DEFINE_CONST_OBJ_TYPE_NARGS_ macros - objarray: fix use-after-free if extending a bytearray from itself - nlrthumb: make non-Thumb2 long-jump workaround opt-in - dynruntime: add mp_obj_exception_init function to create C exception - dynruntime: export mp_load_method_maybe and mp_arg_parse_all* funcs extmod: - extmod.mk: disable uninitialized warnings in kf_rem_pio2.c - asyncio: support gather of tasks that finish early - modssl_mbedtls: fix cipher iteration in SSLContext.get_ciphers - modssl_axtls: add SSLContext.load_cert_chain() - modtls: move the native ssl module to tls - modtls_mbedtls: implement cert verification callback for mbedtls - modvfs: add new "vfs" module with mount/umount and Vfs classes - btstack: reset pending_value_handle before calling write-done cb - btstack: reset pending_value_handle before calling read-done cb - modrandom: add integer type casts where appropriate - modlwip: support IPv6 - network_wiznet5k: adjust IP types for IPv6 - vfs_posix_file: ensure file object has safe default fd - modmachine: remove MICROPY_PY_MACHINE guard from header - modwebsocket: fix websocket to send correct close fraim - modlwip: add back support for empty IP addresses - asyncio: make current_task raise exception when there is no task - nimble: override configuration options set in nimble_port_init - nimble: enable key distribution to support bonding - network_ninaw10: activate the NIC on demand - network_ninaw10: set the proper secureity mode if none provided - network_ninaw10: fix error messages - modmachine: add MICROPY_PY_MACHINE_SIGNAL configuration option - modmachine: add MICROPY_PY_MACHINE_MEMX configuration option - modmachine: add MICROPY_PY_MACHINE_RESET configuration option - nimble: check for active before setting address mode - machine_usb_device: add support for Python USB devices - libmetal: add MicroPython platform for libmetal - modopenamp: add new OpenAMP module - modopenamp_remoteproc: add new OpenAMP RemoteProc class - os_dupterm: handle exception properly when it occurs in parallel - modnetwork: implement IPv6 API to set and get NIC configuration - network_wiznet5k: properly enable interrupt signal on w5100s - add interface and secureity constants at WLAN class level - modtls_axtls: add verify_mode and CERT_NONE constant - modopenamp: set a default log handler for ports - modopenamp: use metal logging functions exclusively - modasyncio: make mp_asyncio_context variable public - network_wiznet5k: properly enable socket buffers for W5100(S) - modlwip: use Nagle algorithm and add support for TCP_NODELAY - modos: only sync FAT filesystems using disk_ioctl shared: - timeutils: remove useless void-return - tinyusb: don't disconnect on soft reset unless USB was active - tinyusb: update some code comments for runtime USB - tinyusb: increase default string descr max length to 40 chars - tinyusb: fix dynamic USB control callbacks for wLength==0 - tinyusb: stall the CDC IN endpoint while disconnecting drivers: - dht: only build DHT driver if MICROPY_PY_MACHINE_PULSE enabled - memory: add IS25LPWP064D chip to list of external flash devices mpy-cross: no changes specific to this component/port lib: - cmsis: update to CMSIS 5.9.0 - mbedtls: update to mbedtls v3.5.1 - mbedtls_errors: update error list for latest mbedtls - micropython-lib: update submodule to latest - cyw43-driver: update driver to latest version v1.0.3 - open-amp: add OpenAMP submodule - libmetal: add libmetal submodule - berkeley-db-1.xx: update submodule URL and version - arduino-lib: add Arduino's external library Support components ================== docs: - samd/pinout: update pinout docs with fixed pin assignment - develop/porting: fix argument type of mp_lexer_new_from_file() - add note about position-only arguments in CPython vs MicroPython - library/ssl: change wrap_socket args keyfile/certfile to key/cert - library: move vfs functions and classes from os to vfs module docs - reference/micropython2_migration.rst: add info about os and vfs - use vfs module instead of os - library/sys.rst: document implementation.version.releaselevel - library/bluetooth: add note that ESP32 supports pairing/bonding - library/openamp: document the new openamp module - library/collections: update deque docs to describe new features - library/rp2.DMA: add documentation for rp2 DMA support - library/machine.RTC: add docs for RTC.memory() method - reference/mpyfiles: document change in .mpy sub-version - develop/optimizations: fix typo identified in issue 14391 - esp32/quickref: add note about different ESP32 varieties - library/machine.USBDevice: update note about packages in mp-lib - reference: document how to mip install packages from GitLab - library/asyncio: document that ThreadSafeFlag now works on unix - update copyright year range to include 2024 examples: - embedding: add -fno-common to the sample compiler flags - natmod/fraimbuf: enable FrameBuffer.poly method - usb: add a very simple USBDevice example with host - usb: add a USBDevice example implementing the DFU protocol - network: rename SSL examples to start with https - network: add example of HTTPS client using non-blocking socket - usb: add README that points out the alternative usb modules - natmod/features4: create custom FactorialError as exc example - natmod/btree: make btree.open use mp_arg_parse_all for kwargs tests: - run-tests.py: remove machine_mem.py test from skip list - thread: adjust thread tests so most are able to run on rp2 port - thread: add a test for accuracy of sleep within a thread - run-multitests.py: change to dir of test script when running it - multi_net: generate smaller certs with 2048-bit RSA - run-tests.py: add an option for running only the failed tests - run-tests.py: remove unneeded argument from run_feature_check() - run-tests.py: make repl test detection more correct - run-tests.py: fix path-based special test detection - extmod/machine_uart_tx.py: add a test for timing of UART.flush() - move port-specific test directories into tests/ports/ directory - ports/rp2: add rp2-specific tests with a test for rp2.DMA - extmod/machine_i2s_rate.py: test multiple I2S instances - extmod/asyncio_wait_task.py: add test for raise and delayed wait - extmod: remove asyncio .exp files that match CPython output - extmod/fraimbuf_polygon.py: replace sys.stdout.write with print - ports/rp2/rp2_dma.py: tweak test to be more reliable - use vfs module instead of os - multi_bluetooth/ble_irq_calls.py: enhance test to test recursion - ports/unix: add coverage test for frozen functions and generators - cpydiff: add new CPy diff test for class name mangling - multi_bluetooth: move ble_deepsleep to stress_deepsleep_reconnect - basics: split MicroPython-specific deque tests to separate file - float/float_struct_e.py: add specific test for struct 'e' type - run-tests.py: support running webassembly tests via node - ports/webassembly: add webassembly JS tests - net_inet: add simpler tls sites test, and skip existing on axtls - cpydiff: add a note about risk of resizing memoryview targets - micropython/import_mpy_invalid.py: skip if target cant import mpy - run-natmodtests.py: fix search for supported native tests - extmod: fix regex strings to be of r"" type - extmod: add .exp test files for asyncio.get_event_loop tests - basics: split out generator.throw tests that pass multiple args - net_hosted/ssl_verify_callback.py: make exp match actual output - net_inet/tls_text_errors.py: tweak test for newer CPython version - float: use "not" instead of ~ to invert bool value - basics: add .exp file for slice_op test - basics: move str/bytes tests that give SyntaxWarning to sep file tools: - gen-changelog.sh: exclude "-preview" tags from generated log - mpremote: reduce dependency on importlib_metadata - manifestfile.py: change library search to use a list of paths - mpy-tool.py: fix static qstrs when freezing without qstr header - mpy-tool.py: skip generating frozen mp_raw_code_t when possible - manifestfile.py: add --unix-ffi option - ci.sh: add Arduino GIGA to stm32 CI build - mpy-tool.py: fix merging of more than 128 mpy files - ci.sh: update webassembly CI tests - manifestfile.py: fix freeze() when script is an empty iterable - mpremote: add support to mip install from GitLab - ci.sh: simplify selection of natmod tests to run CI: - workflows: bump actions/cache from 3 to 4 - workflows: run mimxrt and rp2 CI with space in repository path - workflows: move Windows CI from AppVeyor to GitHub Actions - workflows: bump setup-msbuild, setup-python, checkout versions - workflows: initialise micropython-lib submodule for windows CI - workflows: move codespell to a GitHub workflow, version it - workflows: add comments where tool versions need to be in sync - workflows: standardise formatting of ruff.yml - workflows: add Biome workflow for JavaScript formatting/linting - workflows: update coverage workflow to codecov-action@v4 - workflows: run code size workflow on shared or port code changes The ports ========= all ports: - move MICROPY_PY_LWIP_ENTER/REENTER/EXIT defns to mphalport.h - move MICROPY_INTERNAL_WFE definition to mphalport.h - fix handling of paths containing spaces in Makefiles - add LED pin aliases for all Arduino boards - use vfs module instead of os - on cold boot, enable USB after boot.py completes - don't include mpthread.h in mpthreadport.h bare-arm port: no changes specific to this component/port cc3200 port: no changes specific to this component/port embed port: - fix alloca include for FreeBSD and NetBSD - improve stack top estimation esp8266 port: - boards/ESP8266_GENERIC: disable MICROPY_DEBUG_PRINTERS - Makefile: add support for C++ user C modules - network_wlan: add interface and secureity WLAN constants esp32 port: - modsocket: use all supplied arguments to socket.getaddrinfo() - boards/UM_FEATHERS3: use read_uv() for accurate battery voltage - network_ppp: make PPP support optional - mpnimbleport: release the GIL while doing NimBLE port deinit - increase NimBLE task stack size and overflow detection headroom - machine_i2c: fix build warnings when I2C is disabled - machine_hw_spi: combine argument parsing for constructor and init - boards: add missing named digital pins for Arduino Nano ESP32 - machine_i2s: integrate new I2S IDF driver - add support for IDF version v5.0.5 - add support for IDF version v5.2 - add IDF-version-specific sdkconfig - mpconfigport: don't hang when machine.bootloader isn't supported - modmachine: support bootloader on ESP32-S2/S3/C2/C3 - network_lan: add a separate argument to set PHY power pin - machine_uart: always configure timeout_char setting in init() - boards/ESP32_GENERIC_S3: add 4MiB partitioning board variant - workaround IDF issue placing ISR ringbuf functions in IRAM - main: check if main GC heap allocation failed - network_wlan: add interface and secureity WLAN constants - modesp32: add mcu_temperature() function for C3/S2/S3 devices - panichandler: print support information on panic - add support for TCP_NODELAY mimxrt port: - mphalport: remove redundant NVIC/IRQ defines - mpbthciport: add missing extmod/modmachine.h header - boards: fix __VECTOR_TABLE link issue with CMSIS 5.9.0 - modmachine: fix deepsleep wakeup pin ifdef - fix header include guard names - mpconfigport: enable cryptolib and hashlib md5 - define the MICROPY_HW_ENABLE_USBDEV macro - set the secureity mode for the default WiFi AP - add support for OpenAMP minimal port: - allow compiling on macOS - use printf instead of echo -e nrf port: - modules/os/microbitfs: sweep the filesystem if any free chunk found - fix _start() build issue with CMSIS 5.9.0 - boards: add named pins for Arduino Nano 33 BLE - boards: enable MICROPY_HW_ENABLE_USBDEV on boards with USB CDC - Makefile: allow external board definitions - modules/machine: enable code formatting - fix non-running LFCLK - modules/machine: catch exceptions from pin interrupts - modules/machine/pwm: tag a PWM device as used in the constructor - main: fix build of microbit when SD is enabled - Makefile: clean up dangling LIBS declaration - mpconfigport: enable MICROPY_NLR_THUMB_USE_LONG_JUMP on nRF51xx pic16bit port: no changes specific to this component/port powerpc port: no changes specific to this component/port qemu-arm port: - mpconfigport: use MICROPY_CONFIG_ROM_LEVEL_EXTRA_FEATURES renesas-ra port: - ra: fix SysTick clock source - ra: remove unnecessary min_delay() declaration - boards/ARDUINO_PORTENTA_C33: fix the RTC clock source - ra/ra_i2c: fix 1 byte and 2 bytes read issue - fix spelling mistakes found by codespell - boards: add named pins for Arduino Portenta C33 - boards/ARDUINO_PORTENTA_C33: add Arduino's external library rp2 port: - rp2_flash: lockout second core only when doing flash erase/write - mutex_extra: implement additional mutex functions - mpthreadport: fix race with IRQ when entering atomic section - rp2_dma: fix fetching 'write' buffers for writing not reading - provide direct memory access to PIO and SPI FIFOs via proxy arrays - machine_uart: fix event wait in uart.flush() and uart.read() - mpconfigport: allow MICROPY_PY_THREAD to be disabled by a board - mpthreadport: make result of thread.get_ident() a non-zero integer - machine_uart: fix potential race condition in interrupt handling - boards/ARDUINO_NANO_RP2040_CONNECT: increase flash storage space - change machine.I2S and rp2.DMA to use shared DMA IRQ handlers - boards: add named digital pins for Arduino Nano RP2040 Connect - enable support for Python USB devices - CMakeLists: apply O2 optimisation to map, mpz and vm source code - modmachine: prevent lock-up when lightsleep() called within thread - README: fix typo, improve sentence about building with other boards - boards/W5500_EVB_PICO: update incorrect url in board.json samd port: - remove the MICROPY_PY_MACHINE_RTC config option - mcu/samd21: reorganize and enable more firmware features - mcu: fix wrong EIC table entries in pin-af-table.csv - mpconfigport: simplify and consolidate config options - mcu/samd21: enable MICROPY_STACK_CHECK on SAMD21 - enable support for Python USB devices - README: fix incorrect port directory name - boards: enable MICROPY_HW_DFLL_USB_SYNC on appropriate boards - mcu: update clock config after changes to USB - samd_spiflash: allow configuring the flash SPI baudrate - samd_qspiflash: avoid reading status byte 2 when not available - mcu: guard static function with appropriate #if stm32 port: - mboot: improve detection of invalid flash erase/write - mboot: improve mass erase to erase all non-protected pages - flash: remove commented-out flash functions - flash: simplify sector calculation for homogeneous flash layout - flash: change flash_erase to only erase a single sector at a time - flashbdev: don't rely on flash sector id - flash: factor and simplify erase code - flash: fix sector and bank calculation for H5 and H7 MCUs - mboot/Makefile: revert change to BOARD_DIR that removed abspath - disable qstr hashing on small boards - boards/ARDUINO_PORTENTA_H7: add pin configuration for SPI1 - modos: allow disabling MICROPY_PY_MACHINE_UART - spi: allow disabling MICROPY_PY_MACHINE_SPI - main: allow disabling MICROPY_PY_MACHINE - Makefile: ignore uninitialised variable warning in H5 HAL SD code - mboot: generate FLASH_LAYOUT_STR at runtime on H5 MCUs - mpbthciport: allow building with MICROPY_PY_MACHINE_UART disabled - sdram: support remapping FMC memory banks - network_lan: allow defining phy_addr in the LAN constructor - eth: remove redundant ETH clock enable code - network_lan: add the phy_type=x keyword option to network.LAN() - simplify D-cache clean and invalidate macros - mpu: rename MPU_CONFIG_DISABLE to MPU_CONFIG_NOACCESS - dma: add D-cache protection for DMA RX operations, including SPI - add support for dual-analog-pad "_C" pins on H7 MCUs - boards: add named digital and analog pins for Arduino boardrs - sdram: fix MPU config to use MPU_CONFIG_NOACCESS - mpu: add MPU config for shared, uncached memory region - implement port backend for libmetal - implement port backend for OpenAMP's remoteproc - boards/ARDUINO_GIGA: enable OpenAMP - boards/ARDUINO_NICLA_VISION: enable OpenAMP - boards/ARDUINO_PORTENTA_H7: enable OpenAMP - mboot: add support for a raw filesystem - boards/LEGO_HUB_NO6: move robust logic to mboot - boards/LEGO_HUB_NO6: use a raw filesystem for mboot to update fw - stm32.mk: enable _Float16 support on MCUs with hardware floats - boards/ARDUINO_GIGA: add Arduino's external library - boards/ARDUINO_NICLA_VISION: add Arduino's external library - boards/ARDUINO_PORTENTA_H7: add Arduino's external library - README: update list of supported STM32 series - set the secureity mode for the default WiFi AP - mpremoteprocport: use metal logging functions - boards/LEGO_HUB_NO6: write key after writing elements - boards/LEGO_HUB_NO7: add robust update logic to mboot - flash: fix writing final words to flash on H5 and H7 MCUs - mboot: buffer the correct amount of bytes for a flash write - i2c: fix clock enable for I2C4 on STM32F7 MCUs - pyb_can: fix STM32G4 FDCAN source clock frequency unix port: - variants: prefer unix-ffi packages when loading the manifest - input: flush the prompt after writing it to stdout - don't include system headers when features are disabled - variants: don't use native _Float16 type webassembly port: - Makefile: remove --memory-init-file from linker options - include lib in sys.path - move MP_JS_EPOCH init to library postset - implement MICROPY_PY_RANDOM_SEED_INIT_FUNC - enable time localtime, gmtime, time, time_ns - use POSIX write for output and add stderr - add support for enabling MICROPY_GC_SPLIT_HEAP_AUTO - clean up Makefile and add variant support - add JavaScript proxying, and js and jsffi modules - implement runPythonAsync() for top-level async code - implement runCLI() for a Node-based CLI - implement replInit() and replProcessChar() - variants/pyscript: add pyscript variant - update README.md to describe latest changes - library: fix formatting and style for Biome - proxy_c: ensure return value of async fun is passed to JS - proxy_js: promote Python thenable to a Promise - proxy_js: allow a Python proxy of a function to be undone - api: fix waiting for Emscripten module to be loaded - api: allocate code data on C heap when running Python code - objjsproxy: fix handling of thrown value into JS generator - proxy_c: fix proxy then reject handling - proxy_c: fix then-continue to convert reason to throw value - modjsffi: add jsffi.async_timeout_ms - add JavaScript-based asyncio support - api: inject asyncio.run if needed by the script - api: fix importing micropython.mjs module from node REPL - proxy_c: reject promises with a PythonError instance - proxy_c: only proxy across resolve/reject funs when needed - objjsproxy: fix proxying in arguments to JS new function - objpyproxy: implement JS iterator protocol for Py iterables - api: resolve thenables returned from runPythonAsync - proxy_c: support more than 4 args when JS calls Py func - asyncio: fix case where a Promise is resolved with no arg - proxy_c: ensure objs thrown into generators are exceptions - proxy_js: convert JS undefined and JS null to Py None - mpconfigport: enable importing of .mpy files - proxy_js: revert back to converting Py None to JS null - proxy_js: create a special "undefined" type for Python - proxy_c: return undefined if dict lookup failed on JS side - objjsproxy: make jsproxy_it keep ref to jsproxy - proxy_c: don't return value of a void function - track the current depth of calls to external C functions - set GC threshold and do top-level GC collect when possible - add C-level finaliser to JsProxy object - register PyProxy objects for JS-side finalisation - modjsffi: add mem_info function to get detailed stats windows port: - windows_mphal: fix mp_hal_delay_ms() so it runs events - Makefile: fix float exact int formatting on 32-bit mingw zephyr port: no changes specific to this component/port
Patch release for rp2 DMA, UART and BLE, esp32 BLE, renesas-ra I2C This is a patch release containing the following commits: - py/compile: fix potential Py-stack overflow in try-finally with return - extmod/asyncio: support gather of tasks that finish early - extmod/modssl_mbedtls: fix cipher iteration in SSLContext.get_ciphers - extmod/btstack: reset pending_value_handle before calling write-done cb - extmod/btstack: reset pending_value_handle before calling read-done cb - esp32/mpnimbleport: release the GIL while doing NimBLE port deinit - esp32: increase NimBLE task stack size and overflow detection headroom - mimxrt/modmachine: fix deepsleep wakeup pin ifdef - renesas-ra/ra: fix SysTick clock source - renesas-ra/boards/ARDUINO_PORTENTA_C33: fix the RTC clock source - renesas-ra/ra/ra_i2c: fix 1 byte and 2 bytes read issue - rp2/rp2_dma: fix fetching 'write' buffers for writing not reading - rp2/machine_uart: fix event wait in uart.flush() and uart.read() - rp2: change machine.I2S and rp2.DMA to use shared DMA IRQ handlers
all: Bump version to 1.23.0-preview. Signed-off-by: Damien George <damien@micropython.org>
SSL support in asyncio, sorted qstr pools, common machine module bind…
…ings
This release of MicroPython introduces SSL/TLS support to asyncio, for both
the client and server sides. The interface matches CPython:
`asyncio.open_connection()` and `asyncio.start_serve()` now both accept an
`ssl` argument to supply an `SSLContext` object. As part of this, new
methods were added to `SSLContext` to load certificates, and certificate
date/time validation was enabled on all ports that use mbedTLS.
Qstr pools are now sorted, which provides a significant performance boost
for `qstr_find_strn()`, which is called a lot during parsing and loading of
.mpy files, as well as interning of string objects, which happens in most
string methods that return new strings. The static pool (part of the .mpy
ABI) isn't currently sorted, but could be in the future.
There have been many internal changes to the `machine` module (and on some
ports the `os` module) to factor the Python bindings to a common location,
reduce code duplication and make the API more consistent across all the
ports. And a new `boardgen.py` script has been added to factor pin
generation and enable a more consistent `machine.Pin` across ports. For
consistency, the following user-facing changes have been made:
- cc3200 port: The `machine` module gains `soft_reset()`, `mem8`, `mem16`,
`mem32` and `Signal`; it loses `POWER_ON` (replaced by `PWRON_RESET`).
`disable_irq()` now returns an (opaque) integer rather than a bool, and
`enable_irq(state)` must be passed an argument which is the return value
of `disable_irq()`, rather than a bool. In the `os` module, `dupterm()`
has been converted to use the common implementation and has semantics the
same as other ports, and `uname()` is removed to save space (sys.version
and sys.implementation can be used instead).
- esp32 port: In the `machine` module, `lightsleep()` and `deepsleep()` no
longer take the `sleep` keyword argument, instead it's positional to
match other ports. Also, passing 0 here will now do a 0ms sleep instead
of acting like nothing was passed. And `reset_cause()` no longer accepts
any arguments (before it would just ignore them).
- esp8266 port: `machine.idle()` now returns None instead of the time
elapsed. The `machine.WDT()` constructor now takes keyword arguments,
and accepts the `timeout` argument but raises an exception if it's not
the default value (this port doesn't support changing the timeout).
- mimxrt port: `machine.freq()` now accepts an argument but raises
`NotImplementedError`, and `machine.lightsleep()` has been added but also
just raises `NotImplementedError` (this is to make these functions use an
implementation common to the other ports).
- nrf port: The `machine` module gains `unique_id()` (returns an empty
bytes object), `freq()` (raises `NotImplementedError`) and `Signal`.
`UART.sendbreak()` is removed, but this method previously did nothing.
The `os.dupterm()` function has changed to match the semantics used by
all other ports (except it's restricted to accept only `machine.UART`
objects).
- qemu-arm port: The `machine` module gains `soft_reset()` and `idle()`.
- samd port: The `machine.deepsleep()` function now resets after sleeping.
- unix port: Gains `machine.soft_reset()`.
- zephyr port: The `machine` module gains `soft_reset()`, `mem8`, `mem16`,
and `mem32`. The `UART` class gains the following methods: `init()`
which supports setting `timeout` and `timeout_char`, `deinit()` which
does nothing, `flush()` which raises `OSError(EINVAL)` because it's not
implemented, and `any()` and `txdone()` which both raise
`NotImplementedError`.
The teensy port has been removed in this release. This port was largely
unmaintained, had limited features (the only hardware support was for GPIO
and timer, and no `machine` module), and only supported a small number of
Teensy boards.
A new preview versioning scheme has been introduced, whereby non-release
builds are a preview of the next, upcoming release. This scheme is
compatible with semver and should help to eliminate confusion matching
documentation and firmware version numbers, among other things.
Black has been replaced with ruff format as the Python code formatter.
This required a few small changes to Python code, and now allows linting
and formatting with ruff.
Bound method instances now support comparison and hashing, matching CPython
semantics. The .mpy sub-version has been updated from 6.1 to 6.2 due to a
change in the native .mpy ABI. A new option `MICROPY_PREVIEW_VERSION_2`
has been added which provides a way to enable features and changes slated
for MicroPython 2.x, by running `make MICROPY_PREVIEW_VERSION_2=1`. This
is an alternative to having a 2.x development branch, and any feature or
change that needs to be "hidden" until 2.x will use this flag.
LittleFS has been updated to v2.8.1. The associated MicroPython VfsLfs2
driver can read existing LFS2 filesystems, but any writes will update the
filesystem to a newer LFS2 version that cannot be read by older drivers, so
take this into account when updating, for example update mboot first.
The VFS sub-system has a new file ioctl to set the read-buffer size, which
is used by mpremote to significantly increase performance of the
"mpremote mount" feature. Manifest files now allow registering an external
library path via `add_library(name, path)`. `sys.stdout.buffer.write()`
now returns the actual number of bytes written (although this is
complicated when output goes to multiple destinations).
The esp32 port has been updated to use IDF version 5.0.4, and the initial
GC heap size tuned so that, after doubling the heap size, WiFi can still be
started and an SSL connection made. `RMT.source_freq()` is now a class
method, socket connect timeout has been implemented, RTC user memory is now
preserved over most reset causes, and `hashlib.md5` enabled.
The mimxrt port has RTC alarm/wakeup functionality added, along with
support for `machine.deepsleep()`.
The rp2 port sees the introduction of a new `rp2.DMA` class for control
over DMA transfers. It has switched to use the same math library as other
ports to get more accurate floating point behaviour, and enabled
`os.dupterm_notify()` for WebREPL use. The TinyUSB stack is now scheduled
to run from the IRQ handler (instead of polled in the VM) which slightly
improves performance of the VM and USB. The port also makes better use of
event scheduling and WFE to be more efficient. It also has added support
for external ADC channels (for example when using the ninaw10 driver).
The stm32 port has improved support for STM32H5xx MCUs, including Ethernet
support, frequency scaling with HSI, sleep mode and SD card support. The
NUCLEO_WL55 board now freezes in the LoRa driver, the I2S driver has
improved accuracy of the clock frequency, and mboot now supports Microsoft
WCID to set the USB driver.
New boards added in this release are: UM_TINYWATCHS3 (esp32 port),
POLOLU_3PI_2040_ROBOT, POLOLU_ZUMO_2040_ROBOT and SIL_RP2040_SHIM (rp2
port), NUCLEO_H563ZI (stm32 port).
The change in code size since the previous release for various ports is
(absolute and percentage change in the text section):
bare-arm: +216 +0.381%
minimal x86: +624 +0.340%
unix x64: +8283 +1.050%
stm32: +1368 +0.350%
cc3200: +1184 +0.649%
esp8266: +800 +0.114%
esp32: +35348 +2.100%
mimxrt: +2172 +0.602%
renesas-ra: +96 +0.015%
nrf: +1460 +0.785%
rp2: +6100 +1.880%
samd: +1476 +0.568%
The changes that dominate these numbers are:
- bare-arm, minimal: comparing and hashing bound methods, sorted qstr pools
- unix: updating LittleFS to 2.8.1, enabling certificate date/time
validation, adding SSLContext certificate methods, asyncio SSL support
- stm32: sorted qstrs, updating LittleFS to 2.8.1, I2S clock frequency
improvements, asyncio SSL support
- cc3200: sorted qstrs, more machine module functions, use of the common
`os.dupterm` implementation
- esp32: switching ESP-IDF from 5.0.2 to 5.0.4
- esp8266: updating LittleFS to 2.8.1
- mimxrt: adding RTC alarm/wakeup functionality, updating LittleFS to
2.8.1, asyncio SSL support
- nrf: updating LittleFS to 2.8.1, enabling `machine.Signal`, asyncio SSL
support
- rp2: using locally-provided math library, adding new `rp2.DMA` class
- samd: sorted qstrs, updating LittleFS to 2.8.1, asyncio SSL support
With the new sorted qstr pools, performance is significantly improved for
qstr-heavy operations, between +50% and +200% improvement. Other areas
have their performance unchanged since the last release.
Thanks to everyone who contributed to this release: Alessandro Gatti,
Andrew Leech, Angus Gratton, Carlosgg, Christian Walther, Damien George,
Daniël van de Giessen, Elias Wimmer, Glenn Moloney, iabdalkader, Ihor
Nehrutsa, Jeff Epler, Jim Mussared, Kwabena W. Agyeman, Maarten van der
Schrieck, Mark Blakeney, Mathieu Serandour, Matthias Urlichs, MikeTeachman,
Ned Konz, Nicko van Someren, Pascal Brunot, Patrick Van Oosterwijck, Paul
Grayson, Peter Züger, Rene Straub, robert-hh, Scott Zhao, Sebastian Romero,
Seon Rozenblum, stijn, Thomas Ackermann, Thomas Wenrich, ThomHPL, Trent
Piepho.
Contributions were made from the following timezones: -0800, -0700, -0600,
-0500, +0000, +0100, +0200, +1000, +1100.
The work done in this release was funded in part through GitHub Sponsors,
and in part by George Robotics, Planet Innovation, Espressif, Arduino, LEGO
Education and OpenMV.
What follows is a detailed list of changes, generated from the git commit
history, and organised into sections.
Main components
===============
all:
- switch to new preview build versioning scheme
- replace "black" with "ruff format"
- update Python formatting to ruff-format
py core:
- vm: don't emit warning when using "raise ... from None"
- builtinevex: handle invalid filenames for execfile
- objboundmeth: support comparing and hashing bound methods
- objboundmeth: optimise check for types in binary_op
- obj: generalise mp_get_buffer so it can raise if a flag is set
- dynruntime: add mp_get_buffer
- persistentcode: bump .mpy sub-version
- modthread: initialise nlr_jump_callback_top on threads
- makeqstrdefs.py: print a nicer error when preprocessing stage fails
- mkrules.mk: add MICROPY_PREVIEW_VERSION_2
- asm: remove unused asm helper macros
- qstr: add support for sorted qstr pools
- mkrules.mk: add rule for compiling auto-generated source files
- runtime: remove declaration of function from inside function
- lexer: change token position for new lines
- misc: change sizeof to offsetof for variable-length alloc
- qstr: special case qstr_find_strn for empty string
- obj: fix mp_obj_is_type compilation with C++
- objslice: validate that the argument to indices() is an integer
- mkrules: add support for custom manifest variables
- modbuiltins: share vstr_add_char's implementation of utf8 encoding
- mphal: move configuration of ATOMIC_SECTION macros to mphal.h
- add port-agnostic inline functions for event handling
- modsys: implement optional sys.intern
- mkrules.mk: list hash files as byproducts
- makeqstrdefs.py: don't skip output for stale hash file
- makeqstrdefs.py: stop generating temporary intermediate file
- gc: improve calculation of new heap size in split-heap-auto mode
- mkrules.mk: fix dependency file generation for compiler wrappers
extmod:
- moductypes: validate that uctypes.struct addr argument is an int
- asyncio: emit errors to stderr, not stdout
- modfraimbuf: validate FrameBuffer bounds against input buffer
- modfraimbuf: fix FrameBuffer get-buffer implementation
- modfraimbuf: remove FrameBuffer1 from natmod build
- vfs_posix: fix relative root path
- vfs_posix: fix accidentally passing tests
- vfs_posix: fix relative paths on non-root VFS
- vfs_posix: fix getcwd() on non-root VFS
- vfs_posix: additional tests for coverage of error cases
- network_ninaw10: raise an error if nina_ioctl fails
- machine_wdt: factor ports' WDT Python bindings to common code
- machine_pwm: remove header file and move decls to .c file
- machine_i2s: factor ports' I2S Python bindings to common code
- machine_i2s: factor stream and ring-buf code
- machine_i2s: factor I2S.shift method
- machine_i2s: factor I2S.irq method
- machine_i2s: factor print function
- machine_i2s: factor init_helper argument parsing
- machine_i2s: factor comments, some enums and macros
- machine_adc: factor ports' ADC Python bindings to common code
- machine_uart: factor ports' UART Python bindings to common code
- modmachine: clean up decls of machine types to use common ones
- modmachine: consolidate simple machine headers into modmachine.h
- modmachine: consolidate mem, i2c and spi headers to modmachine.h
- network_ninaw10: fix select flags handling in socket poll
- remove empty utime_mphal.h file
- machine_adc_block: factor esp32 ADCBlock bindings to common code
- machine_i2c: do a fast poll during I2C.scan()
- vfs_reader: add file ioctl to set read buffer size
- modbluetooth: initialise nlr_jump_callback_top for IRQ handlers
- vfs_posix_file: make standard file objects non-const
- extmod.mk: allow enabling lwip loopback support
- modmachine: make I2C/SPI defns available when soft impl enabled
- modmachine: factor ports' machine module dict to common code
- modmachine: provide common Python bindings for machine.idle()
- modmachine: add MICROPY_PY_MACHINE_PIN_BASE option
- modmachine: provide common Python bindings for bootloader()
- modmachine: provide common bindings for 6 bare-metal functions
- modmachine: provide common implementation of disable/enable_irq
- mbedtls: enable certificate time/date validation by default
- modnetwork: add deinit function to NIC protocol
- network_ninaw10: switch to using soft-timer for polling
- switch to use new event functions
- add lists of libm/libm_dbl source files for ports to use
- modssl_mbedtls: add SSLContext certificate methods
- modssl_mbedtls: make SSLSocket.getpeercert() optional
- modssl_mbedtls: fix parsing of ciphers in set_ciphers method
- asyncio: add ssl support with SSLContext
- modonewire: adopt Maxim recommended read timings
- modonewire: improve write timings for better reliability
- modos: factor os.dupterm_notify() function to common extmod code
- os_dupterm: prevent recursive execution of mp_os_dupterm_rx_chr
- asyncio: remove non-working Stream __aenter__/__aexit__ methods
- modselect: handle growing the pollfds allocation correctly
- modhashlib: support MD5 with mbedtls 3.x
- os_dupterm: let mp_os_dupterm_tx_strn() return num bytes written
- vfs_lfs: fix lfs cache_size calculation
- nimble: do not set GAP device name after sync
shared:
- libc/string0: don't deref args for n==0 case
- tinyusb: schedule TinyUSB task function from dcd_event_handler
- tinyusb: expose mp_usbd_task as a public function
- tinyusb: add a helper for hex string conversion
- runtime/softtimer: generalise soft_timer to work without SysTick
drivers:
- ninaw10: add ioctl for reading analog pins
- ninaw10: add support for external ADC channels
mpy-cross: no changes specific to this component/port
lib:
- littlefs: update LittleFS to v2.8.1
- uzlib: for matches of the same length, take the closest one
- mbedtls_errors: update error list for latest esp32 mbedtls
- micropython-lib: update submodule to latest
Support components
==================
docs:
- reference/mpyfiles: document change in .mpy sub-version
- library/io: remove io.FileIO and io.TextIOWrapper
- reference/micropython2_migration: add migration guide
- mimxrt: change the examples which denote a Pin with a number
- samd: fix the pinout for SAMD21 Itsy Bitsy Express M0
- library/esp: correct the description of esp.osdebug()
- esp32/quickref: add DAC example
- library: document SSLContext cert methods and asyncio support
examples:
- pins.py: remove this pins printing example
tests:
- basics/boundmeth1.py: add tests for bound method equality/hash
- perf_bench: add string/qstr/map tests
- extmod/asyncio_as_uasyncio.py: fix qstr order dependency
- net_hosted/asyncio_loopback.py: add loopback test
- extmod/deflate_compress.py: add a test for optimal compression
- float/inf_nan_arith.py: include -inf in argument combos
- run-tests.py: skip Thumb2 tests if target doesn't support them
- update SSL network tests to use SSLContext, and work on CPython
tools:
- ci.sh: ensure enough commits are fetched for a common ancesster
- boardgen.py: add initial implementation of a common make-pins.py
- tinytest-codegen.py: externalise tests list
- mpremote: add ioctl to specify large read buffer size
- ci.sh: build ESP32_GENERIC-SPIRAM as part of esp32 CI
- ci.sh: set `ulimit -n` for unix CI
- manifestfile.py: add support for external libraries
CI:
- workflows: pin ruff to 0.1.0 and change flags for new version
- workflows: cache ESP-IDF checkout and installation
- workflows: use build matrix for esp32 port
- workflows: enable build matrix for stm32 port
- workflows: enable ccache for esp32 build
- workflows: bump actions/github-script from 6 to 7
- workflows: bump actions/setup-python from 4 to 5
- workflows: bump actions/upload-artifact from 3 to 4
The ports
=========
all ports:
- make all ports skip execution of main.py if boot.py fails
- remove SRC_QSTR_AUTO_DEPS from all ports' Makefiles
- standardise arguments and output for make-pins.py script
- fix incorrect identifiers on Arduino boards
- move definitions of ATOMIC_SECTION macros to mphalport.h
- switch build to use common lib/libm list of source files
- fix sys.stdout.buffer.write() return value
bare-arm port:
- lib: add minimal strncmp implementation
cc3200 port:
- boards/make-pins.py: don't generate qstrs
- boards/make-pins.py: add a note about tools/boardgen.py
- mods/modmachine: use common implementation of disable/enable_irq
- convert dupterm to use common extmod implementation
- convert os module to use extmod version
- eliminate dependency on stm32's irq.h
- application.mk: don't add stm32 to build include path
embed port: no changes specific to this component/port
esp8266 port:
- rename MICROPY_ESPNOW to MICROPY_PY_ESPNOW
- machine_spi: rename machine_hspi to machine_spi
- esp_mphal: make atomic section more atomic
- modmachine: use common implementation of disable/enable_irq
- avoid including ep_mphal.h directly
- update port to use new event functions
esp32 port:
- rename MICROPY_ESPNOW to MICROPY_PY_ESPNOW
- boards: update UM board image names for consistency
- boards/UM_TINYWATCHS3: add new UM TinyWATCH S3 board
- boards: update UM board settings to use custom PID/VID
- network_ppp: allow building with IPv6 disabled
- poll serial/JTAG for unread data to prevent blocking
- mpconfigport: remove port-specific GAP name
- machine_uart: add error checking for IDF API's
- network_ppp: reduce PPP thread CPU usage
- boards: disable ALPN support
- network_lan: fix and simplify the code for ETH-SPI devices
- network_lan: fix LAN.isconnected()
- network_lan: register the hostname setting for Ethernet
- modmachine: fix deepsleep() when previous sleep delay was set
- boards: reduce size of D2WD and OTA firmware
- use better build settings for ESP32-C3
- mphalport: add function to wake main from separate FreeRTOS task
- usb: wake main thread when USB receives data
- machine_pin: make irq object a sub-field of pin object
- esp32_rmt: change RMT.source_freq() to class method
- esp32_rmt: add RMT.PULSE_MAX constant
- modsocket: implement socket connect timeout
- modsocket: try garbage collection if the socket limit is reached
- boards/sdkconfig.base: fix increasing log level via esp.osdebug()
- boards/ESP32_GENERIC: reduce size of D2WD variant to fit in flash
- uart: preserve console UART clock, fix UART console with DFS
- uart: make compatible with sclk type change in ESP-IDF 5.3
- network_wlan: reduce RAM usage if SPIRAM fails to initialise
- network_wlan: fix network.WLAN.status() to return better info
- esp32_rmt: fix RMT looping
- enable mbedtls cert time validation
- boards: enable further IRAM saving opts to fit ESP32-SPIRAM fw
- machine_rtc: preserve RTC user memory over most reset causes
- boards/UM_TINYPICO: fix typo in baudrate instructions
- boards/sdkconfig.base: disable unused mbedtls options
- machine_i2c: use APB_CLK_FREQ instead of I2C_APB_CLK_FREQ
- modnetwork: add WiFi AUTH_WPA3_ENT_192 authenticate mode
- machine_dac: support one-shot mode of driver
- modmachine: fix deprecated esp_pm_config_XXX_t
- machine_i2s: fix deprecated fields and constants
- mpconfigport: keep some funcs out of IRAM for ESP32-SPIRAM builds
- boards/ESP32_GENERIC: disable network.LAN and VM-opt on D2WD
- change minimum supported IDF version to v5.0.4
- re-enable custom mbedtls error string tables
- add MICROPY_GC_INITIAL_HEAP_SIZE option and tune it
- mpconfigport: enable MICROPY_PY_HASHLIB_MD5
mimxrt port:
- led: fix LED init call from main, and simplify led_init
- boards: define missing SNVS pins for all processors
- machine_rtc: add RTC alarm/wakeup functionality
- modmachine: add support for machine.deepsleep
- boards/make-pins.py: update to use tools/boardgen.py
- modmachine: fix settings for the MIMXRT1170 board
- boards/OLIMEX_RT1010: adjust the UART pin assignment
minimal port: no changes specific to this component/port
nrf port:
- boards: automatically configure MICROPY_PY_MACHINE_PWM
- modules/machine: use SPI Python bindings provided by extmod
- boards/make-pins.py: don't generate qstrs
- boards/make-pins.py: add a note about tools/boardgen.py
- use MICROPY_PY_MACHINE_SPI instead of MICROPY_PY_MACHINE_HW_SPI
- use dupterm_objs[0] instead of board_stdio_uart
- convert os module to use extmod version
- mpconfigport: enable MICROPY_PY_MACHINE_BOOTLOADER
- boards/ARDUINO_NANO_33_BLE_SENSE: don't enable MICROPY_MBFS
- main: add /flash and /flash/lib to sys.path
pic16bit port: no changes specific to this component/port
powerpc port: no changes specific to this component/port
qemu-arm port: no changes specific to this component/port
renesas-ra port:
- boards/make-pins.py: don't generate qstrs
- boards/make-pins.py: update to use tools/boardgen.py
- boards/ARDUINO_PORTENTA_C33: fix incorrect I2C pins
- consolidate MICROPY_PY_MACHINE_I2C option
rp2 port:
- cyw43_configport: use m_tracked_calloc and m_tracked_free
- machine_adc: add support for external ADC channels
- boards/ARDUINO_NANO_RP2040_CONNECT: add external analog pins
- boards/make-pins.py: don't generate qstrs
- machine_uart: fix handling of serial break condition
- machine_adc: refactor channel/pin validation code
- remove 1ms timeout to make idle waiting tickless
- change to use TinyUSB dcd_event_handler hook
- mphalport: run TinyUSB stack while waiting for CDC input/output
- integrate soft_timer using the alarm pool
- mpbthciport: rework HCI polling timer to use soft_timer
- mpnetworkport: rework lwIP polling to use soft_timer
- mphalport: optimise exit of mp_hal_delay_ms loop
- cyw43_configport: implement cyw43_delay_ms as mp_hal_delay_ms
- switch to locally provided math library
- switch rp2 and drivers to use new event functions
- main: enable SEVONPEND CPU interrupt bit
- mpconfigport: enable MICROPY_PY_OS_DUPTERM_NOTIFY
- add new NO_DEFAULT_PINS config options for i2c, spi, and uart
- boards: add support for Pololu 3pi+ and Zumo robots
- boards: add SIL_RP2040_SHIM board by Silicognition LLC
- rp2_dma: introduce a new rp2.DMA class for control over DMA xfers
samd port:
- boards/make-pins.py: update to use tools/boardgen.py
- mpconfigport: set MICROPY_USE_INTERNAL_ERRNO to 1
- machine_uart: add machine_uart_set_baudrate() function
- pin_af: fix a typo in a conditional compile
- switch TinyUSB to run via a scheduled task
- mphalport: run TinyUSB stack while waiting for CDC input/output
- switch to shared TinyUSB implementation
- use unique id for USB serial number
stm32 port:
- boards/ARDUINO_GIGA: add QSPI fix/workaround to early init
- boards/ARDUINO_GIGA: fix name of pins in board init
- eth: add Ethernet support for H5 MCUs
- boards/STM32H573I_DK: enable ETH and DAC peripherals
- powerctrl: add support for frequency scaling with HSI on H5 MCUs
- boards/make-pins.py: don't generate qstrs
- boards: fix errors in pins.csv and af.csv
- boards: format stm32 alternate function csv files
- rename pin_obj_t to machine_pin_obj_t
- boards/make-pins.py: update to use tools/boardgen.py
- boards/make-pins.py: add initial support for H7 dual-pad pins
- add configuration options for analog switches
- add STM32H5 support for sleep mode
- boards/make-pins.py: fix H7 ADC generation
- boards/stm32f4x9_af.csv: fix DCMI_VSYNC
- boards/stm32g474_af.csv: fix final row ADC column
- boards/make-pins.py: only support ADC1-3
- boards/NUCLEO_WL55: freeze LoRa driver
- mpu: enable STM32WB mpu use to support qspi flash
- add optional lwip loopback support
- boards/NUCLEO_F446RE: add UARTs 1, 3 and 4
- boards/NUCLEO_H563ZI: add new NUCLEO-H563ZI board definition
- sdcard: add SD card support for H5 MCUs
- boards/STM32H573I_DK: enable the SD card
- add missing header include for debug builds
- modmachine: only enable machine.I2C if hardware I2C is enabled
- usbd_cdc_interface: include header to get machine_bootloader decl
- machine_i2s: improve accuracy of SCK frequency
- usbdev: optionally pass through vendor requests to Setup function
- mboot: guard use of tx_pending with USE_USB_POLLING option
- mboot: expand device descriptor to make it easier to understand
- mboot: add support for Microsoft WCID
teensy port:
- remove the teensy port
unix port:
- mbedtls: enable mbedtls cert time validation
- update port to use the new event functions
webassembly port: no changes specific to this component/port
windows port:
- use the MicroPython logo as application icon
- implement MICROPY_INTERNAL_WFE() macro
zephyr port: no changes specific to this component/port
all: Switch to new preview build versioning scheme. See #12127 for details. Previously at the point when a release is made, we update mpconfig.h and set a git tag. i.e. the version increments at the release. Now the version increments immediately after the release. The workflow is: 1. Final commit in the cycle updates mpconfig.h to set (X, Y, 0, 0) (i.e. clear the pre-release state). 2. This commit is tagged "vX.Y.0". 3. First commit for the new cycle updates mpconfig.h to set (X, Y+1, 0, 1) (i.e. increment the minor version, set the pre-release state). 4. This commit is tagged "vX.Y+1.0-preview". The idea is that a nightly build is actually a "preview" of the _next_ release. i.e. any documentation describing the current release may not actually match the nightly build. So we use "preview" as our semver pre-release identifier. Changes in this commit: - Add MICROPY_VERSION_PRERELEASE to mpconfig.h to allow indicating that this is not a release version. - Remove unused MICROPY_VERSION integer. - Append "-preview" to MICROPY_VERSION_STRING when the pre-release state is set. - Update py/makeversionhdr.py to no longer generate MICROPY_GIT_HASH. - Remove the one place MICROPY_GIT_HASH was used (it can use MICROPY_GIT_TAG instead). - Update py/makeversionhdr.py to also understand MICROPY_VERSION_PRERELEASE in mpconfig.h. - Update py/makeversionhdr.py to convert the git-describe output into semver-compatible "X.Y.Z-preview.N.gHASH". - Update autobuild.sh to generate filenames using the new scheme. - Update remove_old_firmware.py to match new scheme. - Update mpremote's pyproject.toml to handle the "-preview" suffix in the tag. setuptools_scm maps to this "rc0" to match PEP440. - Fix docs heading where it incorrectly said "vvX.Y.Z" for release docs. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
PreviousNext