主要改进:
Improved mpy format with support for native code, and new JavaScript port
In this release the mpy file format has been moved to version 4 and has some significant improvements: mpy file size is reduced on average by about 35%, loading time of mpy files is reduced by about 40%, and they now have support to save native, viper and inline assembler code (or machine code generated from any other source). Size reduction of mpy files was achieved by adding a qstr window to reuse past qstrs when encoding them, by packing qstrs directly in the bytecode, and by defining a static qstr set.
Some VM opcodes were also changed to fix a bug when doing a break/continue out of a finally block, and to make some simplifications. In particular POP_BLOCK and POP_EXCEPT opcodes were replaced with POP_EXCEPT_JUMP.
Most uppercase macros have been converted to lowercase to make a more consistent C API, including all MP_OBJ_IS_xxx and MP_xxx_SLOT_IS_FILLED macros.
The default PYTHON makefile variable is now changed from "python" to "python3", but Python 2 is still supported via "make PYTHON=python2".
The mpy-cross compiler supports the new mpy version 4 and has new command line options: "-march=" to select the native emitter, and "--version" to print the MicroPython version and the mpy version. Also mpy-tool.py has support for freezing native code.
A module system for external, user C modules has been implemented and documentation for this is available in the new "docs/develop" section.
A new "javascript" port has been added which targets JavaScript as the machine via Emscripten. This allows to run MicroPython as an application within node.js, and to run it within a browser (among other things).
All bare-metal ports have the following improvements: machine.sleep() is now machine.lightsleep(), and both lightsleep() and deepsleep() now take an optional argument which is the maximum time to sleep in milliseconds. These ports also now allow freezing of boot.py and main.py using the usual methods. And a new I2C method i2c.writevto(addr, vect) is added which can be used to write a tuple/list of buffers all at once to an I2C device.
The stm32 port now has a fully integrated Ethernet MAC driver (see the network.LAN class) using lwIP for the TCP/IP stack, and sockets were made significantly more robust. Support for F413 MCUs was added. There are also some minor user-facing changes to this port:
machine.UART (and pyb.UART) now defaults to timeout=0 (was 1000ms) which gives UARTs a non-blocking behaviour. The USB REPL is now configurable using uos.dupterm, and by default pyb.USB_VCP(0) is put on dupterm slot 1 before boot.py is executed. To disable the default REPL use: uos.dupterm(None, 1). pyb.DAC(id) will now only reset the DAC the first time it is called; to get the old behaviour pass the bits parameter like: pyb.DAC(id, bits). pyb.DAC.noise() and pyb.DAC.triangle() now output at full scale. The system will not recreate boot.py if it's missing (it will only create it if the filesystem is corrupt, or when doing a factory reset).
For the esp32 port, the build process has been updated to align better with the ESP IDF and now uses sdkconfig to configure features. Dual core mode is now enabled by default, SPIRAM is in memory-mapped mode so all of it can be used for the MicroPython heap, there is support to change the CPU frequency, and the WDT now panics and resets the device if it times out.
A detailed list of changes follows.
py core:remove calls to file reader functions when these are disabled add optional support for 2-argument version of built-in next() compile: swap order of pop_block/pop_except in "except as" handler warning: support categories for warnings builtinhelp: only print help re FS modules if external import enabled downcase all MP_OBJ_IS_xxx macros to make a more consistent C API downcase MP_xxx_SLOT_IS_FILLED inline functions mkenv.mk: change default PYTHON variable from "python" to "python3" qstr: evaluate find_qstr only once then pass to Q_GET_HASH macro obj.h: remove obsolete mp_obj_new_fun_viper() declaration objfun: make fun_data arg of mp_obj_new_fun_asm() a const pointer eliminate warnings about unused arguments when debugging disabled compile: fix handling of unwinding BaseException in async with compile: add optimisation to compile OrderedDict inplace objexcept: fix hash of exc str created in mp_obj_new_exception_msg py.mk: update lwip build config to work with latest lwip version fix VM crash with unwinding jump out of a finally block vm: remove currently_in_except_block variable replace POP_BLOCK and POP_EXCEPT opcodes with POP_EXCEPT_JUMP persistentcode: add a qstr window to save mpy files more efficiently persistentcode: pack qstrs directly in bytecode to reduce mpy size persistentcode: define static qstr set to reduce size of mpy files add independent config for debugging sentinel object values emitnative: consolidate where HASCONSTS is set to load-const-obj fun emitnative: provide concentrated points of qstr emit emitnative: adjust accounting of size of const_table emitglue: remove union in mp_raw_code_t to combine bytecode & native add support to save native, viper and asm code to .mpy files persistentcode: bump .mpy version to 4 allow registration of modules at their definition implement a module system for external, user C modules update and rework build system for including external C modules move mp_native_type_from_qstr() from emitnative.c to nativeglue.c nativeglue: rename native convert funs to match other native helpers compile: add support to select the native emitter at runtime compile: support multiple inline asm emitters compile: check that arch is set when compiling native, viper or asm runtime: remove long-obsolete MICROPY_FSUSERMOUNT init code scheduler: convert micropythyon.schedule() to a circular buffer nlrthumb: add support for iOS where the C func is _nlr_push_tail makedefs: use io.open with utf-8 encoding when processing source runtime: optimise to not create temp float for int to power negative runtime: fix mp_unpack_ex so seq can't be reclaimed by GC during use mpprint: support printing %ld and %lu formats on 64-bit archs asmthumb: support asm_thumb code running on normal ARM processors native: improve support for bool type in viper functions remove "if (0)" and "if (false)" branches objgenerator: fix handling of None passed as 2nd arg to throw() objgenerator: remove unneeded forward decl and clean up white space misc.h: rename _MP_STRINGIFY to not use leading underscore in ident persistentcode: change "len" type to size_t for mp_obj_str_get_data objarray: add support for memoryview.itemsize attribute objarray: add decode method to bytearray update makefiles to use $(CAT) variable instead of hard coded "cat" update makefiles to use $(TOUCH) instead of hard coded "touch"
extmod:
moduzlib: update to uzlib 2.9.2 modlwip: add support for polling UDP sockets for writability moduhashlib: include implementation of sha256 only when required convert legacy uppercase macro names to lowercase moduwebsocket: refactor websocket to uwebsocket modlwip: change #ifdef to #if for check of MICROPY_PY_LWIP modlwip: fix bug when polling listening socket with backlog=1 modlwip: add concurrency protection macros modwebrepl: fix logic to handle a put of file of size 0 vfs_fat: update for new oofatfs version modlwip: fix case where concurrency lock isn't released on error moduselect: adjust select_select and poll_register to use size_t vfs_fat: fallback to FAT32 if standard FAT16/SFD format fails modlwip: handle case of connection closing while on accept queue modlwip: handle case of accept callback called with null PCB modlwip: protect socket.accept with lwIP concurrency lock modlwip: free any stored incoming bufs/connections on TCP error modlwip: use correct listening socket object in accept callback modlwip: abort TCP conns that didn't close cleanly in a while modurandom: add init method to seed the Yasmarang generator machine_signal: fix fault when no args are passed to Signal() modussl_mbedtls: support non-blocking handshake modussl_axtls: add non-blocking mode support moducryptolib: add optional AES-CTR support modujson: handle parsing of floats with + in the exponent machine_i2c: change C-level API to allow split I2C transactions machine_i2c: remove need for temporary memory in writemem() call machine_i2c: add i2c.writevto() that can write a vector of bufs modlwip: free any incoming bufs/connections before closing PCB modlwip: register TCP close-timeout callback before closing PCB
lib:
utils/pyexec: implement paste mode with event driven REPL utils/printf: exclude __GI_vsnprintf alias for gcc 9 and above utils/gchelper_m3: add license header and clean up code utils/gchelper_m3: add gc_helper_get_sp() function utils/gchelper: add gchelper.h header file for assembler functions netutils: add function to print tracing info for Ethernet frames oofatfs: update oofatfs library to R0.13c working branch oofatfs: update ffconf.h config for new oofatfs version oofatfs: update oofatfs library to fix issue with logic not stm32lib: update library to fix F7 MMC capacity calculation utils/pyexec: add pyexec_file_if_exists() helper function utils: make pyexec_file_if_exists run frozen scripts if they exist utils/interrupt_char: invalidate interrupt char at start up stm32lib: update library to fix UART9/10 baudrate on F4 MCUs nrfx: upgrade nrfx to v1.7.1 plus a UART bug fix
drivers:
memory/spiflash: rework wait_sr to fix uninit'd variable 'sr' display/ssd1306.py: change to use new i2c.writevto() method
tools:
mpy-tool.py: add support for freezing native code upip.py: use "raise arg" instead of no-arg raise form, for native pyboard.py: add missing line from example usage comments mpy-tool.py: adjust use of super() to make it work with Python 2 mpy-tool.py: fix init of QStrWindow, and remove unused variable pyboard.py: don't accumulate output data if data_consumer used upip.py: add support for multiple index URLs with custom default
tests:
basics: add tests for try-except-else and try-except-else-finally run-tests: support running native tests via mpy import: add test for importing x64 native code extmod: add test for FAT filesystem on a very large block device run-tests: ignore exception in process kill when ending repl test micropython: add some tests for failed heap allocation skip tests needing machine module if (u)machine doesn't exist ussl_basic: disable setblocking() calls basics/sys1.py: add test for calling sys.exit() without any args pyb: update UART expected output now that default timeout is 0 basics: add coverage tests for memoryview attributes
mpy-cross:
enable building of x64 native .mpy files add "-march=" option to select native emitter support compiling with MICROPY_PY___FILE__ enabled automatically select ARMV6 arch when running on such a host add --version command line option to print version info
all ports:
change PYB message prefix to MPY convert legacy uppercase macro names to lowercase update to work with new oofatfs version {stm32,esp8266}: set mpy-cross native arch for frozen native code convert to use pyexec_file_if_exists() to execute boot/main.py
unix port:
mpthreadport: add thread deinit code to stop threads on exit mpthreadport: cleanup used memory on thread exit mpthreadport: remove busy wait loop in thread garbage collection modmachine: handle repeated /dev/mem open errors modffi: eliminate unused-argument warning when debugging disabled Makefile: update coverage tests to match those in Travis mpthreadport: use named semaphores on Mac OS X gcollect: make sure stack/regs get captured properly for GC coverage: add test for printing literal % character modusocket: fix use of setsockopt in usocket.settimeout impl modusocket: raise ETIMEDOUT when connect or accept has timeout
windows port:
fix line wrapping behaviour on the REPL
qemu-arm port:
rework to run bare-metal on boards with Cortex-M CPUs use gchelper code to get registers for GC scanning
stm32 port:
implement machine.lightsleep() rtc: check RTCEN=1 when testing if RTC is already running on boot usbdev: add USB config option for board being self powered usbdev: add USB config option for max power drawn by the board main: make board-defined UART REPL use a static object and buffer boards/stm32f429_af.csv: fix typos in UART defs Tx->TX and Rx->RX mboot: add option to autodetect the USB port that DFU uses sdcard: don't use SD clock bypass on F7 MCUs usb: add flow control option for USB VCP data received from host boards/make-pins.py: add basic support for STM32H7 ADC periphs boards/stm32h743_af.csv: add ADC entries to pin capability table adc: add basic support for ADC on a pin on STM32H7 MCUs mboot: use USB HS as main USB device regardless of USB_HS_IN_FS mboot: add support for STM32F769 MCUs mboot: add support for GPIO ports G, H, I and J mboot: allow deploying via deploy-stlink usb: use USB HS as main USB device regardless of USB_HS_IN_FS usbd_conf: fully support USB HS with external PHY modmachine: make bootloader() enter custom loader if it's enabled boards/STM32F769DISC: support the use of USB HS with external PHY boards/STM32F769DISC: configure for use with mboot by default systick: rename sys_tick_XXX functions to systick_XXX systick: make periodic systick callbacks use a cyclic func table pendsv: clean up pendsv IRQ handler and eliminate duplicate code pendsv: add ability to schedule callbacks at pendsv IRQ level systick: provide better compile-time configurability of slots modnetwork: change lwIP polling to be based on background systick move gchelper assembler code to lib/utils for use by other ports gccollect: use gchelper.h header instead of explicit declaration pendsv: fix inline asm constant and prefix with # character mboot/Makefile: support specifying BOARD_DIR for custom board boards/NUCLEO_L476RG: add support for RNG, DAC and CAN1 boards/make-pins.py: add cmdline options to support use by mboot mboot/Makefile: generate all pin header files from board pins.csv mboot/mphalport.h: include genhdr/pins.h for access to pin names qspi: use static af functions for pin configuration mboot: add support for loading gzip'd firmware from a filesystem mboot: move some BSS vars to new section that isn't zeroed out mboot: add support script which can program mboot and application modmachine: add ability to pass through user data to mboot mboot: add hook to run board-specific code early on startup boards/stm32f429.ld: increase uPy heap size by 64k for F429 MCU extint: fix ExtInt to work with non-GPIO pins extint: fix RTC Alarm/FS USB EXTI constants for L4 make-stmconst.py: improve regex to parse more constants extint: add non-GPIO EXTI IRQ sources for F0 extint: remove unused (and incorrect) EXTI defines stm32_it: fix RTC IRQ handler to handle all EXTI IRQs on F0 MCUs sdram: increase GPIO speed for SDRAM interface to "very high" boards/NUCLEO_F767ZI: fix up comments about HCLK computation mphalport: add mp_hal_get_mac() helper function eth: add low-level Ethernet MAC driver network_lan: add high-level network.LAN interface to ETH driver mpconfigport.h: enable lwIP concurrency protection mechanism modnetwork: don't call NIC callback if it's NULL boards/NUCLEO_F767ZI: enable lwIP and Ethernet peripheral boards/STM32F7DISC: enable lwIP and Ethernet peripheral boards/STM32F769DISC: enable lwIP and Ethernet peripheral boards/NUCLEO_F429ZI: enable lwIP and Ethernet peripheral qspi: enable sample shift and disable timeout counter Makefile: allow a board to specify its linker sections for FW boards/STM32F769DISC: use external QSPI flash to store some code use global lwip build config and support building without lwip boards: update to use new build config for lwip component add compile-time option to use HSI as clock source mboot: update to match latest oofatfs version allow to build with threading with the GIL disabled qspi: set pin speed to very-high and allow to config some options usb: allow to override USB strings & VID/PID in app and mboot stm32_it: guard UART7_IRQHandler with check for UART7 define mboot: set USE_MBOOT=1 by default in the Makefile system_stm32: provide default value for HSI calibration mpconfigport.h: remove malloc/free/realloc helper macros boards/STM32L476DISC: enable servo support on STM32L476DISC board Makefile: allow to override CROSS_COMPILE with included Makefile make default USB_VCP stream go through uos.dupterm for main REPL timer: expose the PWM BRK capability of Timer 1 and 8 uart: handle correctly the char overrun case of RXNE=0 and ORE=1 rename MICROPY_HW_HAS_SDCARD to MICROPY_HW_ENABLE_SDCARD add support for MMC driver, exposed via pyb.MMCard class moduos: allow to compile again without USB enabled network_wiznet5k: add ability to trace Ethernet TX and RX frames network_wiznet5k: add ability to set the MAC address network_wiznet5k: automatically set MAC if device doesn't have one i2cslave: add support for H7 MCUs usbd_conf: add support for USB HS peripheral on H7 MCUs mboot: add support for H7 MCUs, with H743 flash layout boards/NUCLEO_H743ZI: add config options to support mboot boards/NUCLEO_L432KC: disable complex nos and default frozen mods rtc: add auto-LSE-bypass detection with fallback to LSE then LSI rtc: remove unused LSE detection code rtc: remove non-ASCII mu-character from source code comment timer: correctly initialise extended break settings on F7/H7/L4 system_stm32f0: enable PWR clock on startup system_stm32f0: add support for using HSE and PLL as SYSCLK powerctrl: enable EIWUP to ensure RTC wakes device from standby dac: rework DAC driver to use direct register access usbdev: make USB device descriptors at runtime rather than static usb: add USB device mode for VCP+VCP without MSC powerctrl: deselect PLLSAI as 48MHz src before turning off PLLSAI usb: remove mp_hal_set_interrupt_char now that it's reset at boot main: increase default UART REPL rx buffer from 64 to 260 bytes powerctrl: support changing frequency when HSI is clock source flash: fix bug computing page number for L432 page erase add support for F413 MCUs boards: add NUCLEO_F413ZH board configuration machine_uart: change default UART timeout to 0, for non blocking move factory reset files and code to separate source file usbd_cdc_interface: don't retransmit chars if USB is reconnected rtc: allow overriding startup timeouts from mpconfigboard dma: initialise all members of DMA structs for H7 MCUs dma: always reset and configure the H7 DMA peripheral spi: enable SPI IRQs and add IRQHandlers for H7 MCUs irq: fix IRQ_ENABLE_STATS stats config to work on all MCUs boards/NUCLEO_H743ZI: enable SPI3 on this board modmachine: create dedicated asm function to branch to bootloader adc: fix VBAT_DIV to be 4 for STM32F411 machine_i2c: update to support new C-level I2C API i2c: make timeout for hardware I2C configurable machine_i2c: simplify ROM initialisation of static HW I2C objects i2c: support setting the I2C TIMINGR value via keyword arg sdcard: add switch break to ensure only correct SD/MMC IRQ is run sdram: update MPU settings to block invalid region, change attrs mboot/README: fix some typos, describe bootloader and fwupdate.py modmachine: in bootloader() disable caches before reset of periphs
cc3200 port:
mods/modussl: fix ca_certs arg validation in mod_ssl_wrap_socket modmachine: rename machine.sleep to machine.lightsleep use common gchelper_m3.s code from lib/utils mpconfigport.h: disable compiler optimisation of OrderedDict
esp8266 port:
modmachine: implement optional time_ms arg to machine.deepsleep modmachine: rename machine.sleep to machine.lightsleep modmachine: implement simple machine.lightsleep function modmachine: in lightsleep, only waiti if wifi is turned off modmachine: call ets_event_poll after waiti in machine.idle modmachine: handle overflow of timer to get longer periods fix ticks_ms to correctly handle wraparound of system counter
esp32 port:
don't put py/scheduler.o in iRAM, it's no longer needed Makefile: put all IDF compenents in .a libs to use IDF ld script update to latest ESP IDF using sdkconfig and new ldgen procedure machine_timer: deinit all active timers on soft reset Makefile: make sure that directory exists for sdkconfig.h boards: use auto xtal freq config instead of default 40MHz modsocket: fix crashes when connect/bind can't resolve given addr modmachine: rename machine.sleep to machine.lightsleep use SPIRAM in mem-map mode so all of it can be used for uPy heap modmachine: add support for changing the CPU frequency modsocket: fix usocket.send to accept anything with buffer proto modsocket: change socket.socket to be socket type rather than fun network_lan: add arg to constructor to set clock mode for ETH PHY modnetwork: catch and report Ethernet events network_lan: make power arg to constructor optional modnetwork: fix wifi.isconnected to return False after disconnect modnetwork: implement RSSI for WiFi STA via WLAN.status('rssi') modnetwork: remove redundant esp_log include Makefile: add some missing IDF source files to bootloader and app mphalport: use ets_delay_us for mp_hal_delay_us_fast machine_pin: make it so None as pull value disables pull up/down machine_pin: add new PULL_HOLD pin pull mode machine_pin: rework pull mode config to fix GPIO hold feature mpthreadport: exit vPortCleanUpTCB early if threading not init'd boards: enable dual core support by default boards/sdkconfig: disable WDT check of idle task on CPU1 README: add info about pyparsing and the correct Python version machine_wdt: add timeout arg to select interval, make WDT panic modnetwork: change type to size_t for uses of mp_obj_str_get_data machine_uart: implement UART.deinit() method network_ppp: add a timeout for closing PPP connection machine_touchpad: use HW timer for FSM to enable wake-on-touch modsocket: raise EAGAIN when accept fails in non-blocking mode
javascript port:
add new port targeting JavaScript via Emscripten fix Emscripten async load, and to compile with modern clang library: use Buffer.alloc() since new Buffer() is deprecated pass (error) exit value out from script to process caller Makefile: fix unrepresentable float error by using clamp library: print data as raw bytes to stdout so unicode works
nrf port:
modmachine: rename machine.sleep to machine.lightsleep uart: use formula instead of switch for baudrate calculation uart: remove unused machine.UART() parameters shrink " does not exist" error messages pin: cleanup Pin.str to print relevant information pin: print pull information in Pin.str ticker: remove duplicate NRFX_IRQ_PRIORITY_SET timer: fix disabling Timer 1 when using soft PWM readme: update link to nrfjprog download bluetooth: improve advertisment behavior for nrf52 targets bluetooth: resolve compilation warning in ble_drv.c readme: update make flash command when defining board pwm: remove superfluous NULL in machine_hard_pwm_instances bluetooth: add support for SoftDevice s132 version 6.1.1 board: migrate all nrf52832 targets to new BLE stack bluetooth: deprecate use of SoftDevice s132 v6.0.0 bluetooth: add support for SoftDevice s140 version 6.1.1 board: migrate nrf52840 target to new BLE stack bluetooth: deprecate use of SoftDevice s140 v6.0.0 readme: add section about LTO Makefile: update to match latest oofatfs version boards: add support for BLYST Nano module based boards change types to size_t for all uses of mp_obj_str_get_data machine/i2c: update to support new C-level I2C API nrfx_glue: adapt to nrfx v.1.7.1 uart: change UART driver to be non-blocking and use IRQs mpconfigport.h: enable MICROPY_KBD_EXCEPTION by default uart: make UART print output something, and add write method mphalport: use wfi to save power while waiting at the UART REPL
zephyr port:
modzephyr: revamp stacks_analyze() call modzsensor: rename "TEMP" sensor channel to "DIE_TEMP" prj_minimal.conf: switch to CONFIG_STDOUT_CONSOLE Makefile: proxy ram_report, rom_report targets from Zephyr prj_frdm_k64f.conf: add fxos8700 sensor prj_frdm_kw41z.conf: add new board configuration i2c: add support for hardware i2c CMakeLists.txt: set AR to point to the Zephyr toolchain exe machine_i2c: update to support new C-level I2C API switch back to enabling I2C in board-specific configs
docs:
machine: change sleep to lightsleep and add timeout arguments convert all cases of machine.sleep to machine.lightsleep ure: fix match.group signature to indicate index param is required uos: document extra requirements on stream objs passed to dupterm pyboard: add link to pyboard v1.1 schematic and layout PDF pyboard: make pyboard v1.1 pinout the default shown in quickref develop: fix typos in C-module example for example_add_ints library/machine.Pin: add PULL_HOLD constant to possible pin pulls esp32: add a note to quickref about use of Pin.PULL_HOLD esp32: add example for pin isolation in combination with deepsleep develop: remove paragraph that was copied in error from other doc cmodules: fix example to globally define MODULE_EXAMPLE_ENABLED cmodules: note the various ways MODULE_EXAMPLE_ENABLED can be set esp8266: add tutorial for APA102 LEDs esp32: correct quickref for ESP32 hardware SPI with non-default IO machine.I2C: add writevto method to write a vector of byte bufs pyboard/quickref: refer to new machine.I2C instead of old pyb.I2C
travis:
update to use Ubuntu 16.04 Xenial for CI builds enable test for running native code via mpy