Bootstrapping esp8266 development

August 2019

This article is a short tutorial on programming and using the esp8266ex commonly known, and from here on referred to, as the esp8266. The reason for writing this is to more easily remember what connections need to be done for the different boot modes and how to program the microcontroller. For the purpose of this tutorial, I will be updating the standard AT firmware that usually comes with the microcontroller.

The article assumes that you are using the esp8266 on a classic ESP-01 PCB that exposes a total of 8 pins. For additional info check out espressif.com which provides documentation, datasheets and reference manuals.

Boot options

The best description of the boot modes comes from the esp8266 and esptool Github wiki pages. There are three modes - one for programming and two for booting from external devices. On the classic PCB the MTDO/GPIO15 pin is grounded, limiting the chip to only boot from SPI flash. Thus the two remaining modes are

  • boot from spi flash - GPIO0 high and GPIO2 high
  • program over uart - GPIO0 low and GPIO2 high

Based on the information from the esptool Github wiki page, GPIO2 should always use a pullup resistor to VCC, not a direct connection. The reason is that the pin is configured as an output by the boot ROM.

After connecting the GPIO0 and GPIO2 pins, one will notice that the microcontroller still does not respond to either AT commands or esptool. The reason is that the microcontroller is in low current mode. The CH_EN PIN (denoted CHIP_EN in the official datasheet) needs to be set high in order for the microcontroller to run.

The firmware

The first step towards updating is to obtain the firmware. At the time of writing this article I found two versions of the AT firmware available for the esp8266. The first is based on the non-OS SDK and the second is based on the IDF.

Firmware based on the non-OS SDK

There are two main SDKs availables for the esp8266, namely the non-OS and the RTOS SDKs. For a list of differences between the two SDKs read this post (slightly dated) and the Roadmap section in the RTOS SDK Readme.md file.

The original AT firmware is based on the non-OS SDK. It is distributed as a binary together with SDK releases, as well as on the espressif.com website. Even though the Github page says source code is provided for each release, the archive also contains libraries, binaries, documentation and example code. The version numbers do not match directly, however one can notice that version 3.0 of the non-OS SDK contains version 1.7.0.0 of the AT firmware which presumably matches version 1.7.0 of the firmware on the espressif.com website.

Upon downloading either packageinside the bin directory, we find two at and at_sdio directories, together with some common binaries. The difference between the at and at_sdio folders (and the binaries inside) is explained in this Github issue. Basically, the at version uses UART to send AT commands, whereas the at_sdio version uses SDIO.

The provided binaries are built for specific sizes of the flash chip. Version 1.6.2 of the firmware was the last one to come provide support by default for 1MB flash chips. Starting from version 3.0, the provided binaries target only 2MB or 4MB flash chips.

Firmware based on the IDF

In addition to the classic firmware, there is another version which implements AT commands in the espressif/esp32-at repository. The firmware is labeled version 2.0.0.0_esp8266 in the releases page. The espressif.com website also provides binaries for this firmware labeled ESP8266 IDF AT Bin V2.0.

This firmware is newer than the non-OS firmware. Based on the release notes provided in the zip package, there are some breaking changes between the two versions. It is recommended that existing projects do not update directly from the non-OS firmware to the IDF firmware, but new projects should start with the IDF firmware directly.

Programming

Programming the esp8266 microcontroller is done with the official esptool application from espressif. The tool talks directly to the bootloader running from the ROM of the microcontroller.

The ROM only listens for esptool commands in the beginning, for a short period of time. If programming is attempted too late, then esptool complains that it cannot connect to the chip. In order to fix this, I run an esptool command and while it tries to connect I reset the microcontroller by pulling reset pin low.

Another common issue that I encountered are timeouts. I believe this happens from several resets mixed with flash writes. The best way to fix this issue is to do a hard reset of the chip by completely powering it off.

As previously mentioned there are multiple binaries distributed depending on the size of the flash chip present on the PCB. To determine the size of the flash chip esptool has the following command:

esptool.py --chip esp8266 --port /dev/ttyUSB0 --baud 115200 flash_id

which will have an output similar to:

esptool.py v2.7
Serial port /dev/ttyUSB0
Connecting...
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: 5c:cf:7f:14:1a:50
Uploading stub...
Running stub...
Stub running...
Manufacturer: e0
Device: 4014
Detected flash size: 1MB
Hard resetting via RTS pin...

In the case above, the size of the flash is 1MB. This means that I can only flash version 2.2.1 of the AT firmware on this device. From the README.md file in the bin/at folder, we see that we should flash the binaries according to the 8Mbit ( 512KB + 512KB ) configuration. The addresses are in the aforementioned README.md file. To flash the binaries with esptool run the following command:

esptool.py --chip esp8266 --port /dev/ttyUSB0 --baud 115200 write_flash 0x00000 boot_v1.7.bin

which will have an output similar to:

esptool.py v2.7
Serial port /dev/ttyUSB0
Connecting....
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: 5c:cf:7f:14:1a:50
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Auto-detected Flash size: 1MB
Flash params set to 0x0020
Compressed 4080 bytes to 2936...
Wrote 4080 bytes (2936 compressed) at 0x00000000 in 0.3 seconds (effective 123.0 kbit/s)...
Hash of data verified.
Leaving...
Hard resetting via RTS pin...

To test the new firmware, open minicom and run AT commands from the official manual directly. The microcontroller expects commands to be terminated with the \r\n sequence. To achieve this in minicom with default settings, press ENTER (for \r) and then CTRL+J (for \n). For example, to get a list of available access points, run the commands

  • AT+CWMODE_CUR=1 (or just AT+CWMODE=1 in older versions) to set the microcontroller in station mode
  • AT+CWLAP to list the available access points

Bonus

The esp8266 uses the xtensa architecture, which requires a non-standard compiler toolchain. The toolchain is not provided by any linux distribution, but can obtained from espressif. If your host is running an x86 or amd64 processor, you can use a pre-compiler toolchain from here. Otherwise if you need to compile the toolchain yourself, the steps are here. Both approaches worked for me smoothly.

From there on you can use the quick start guide, the IoT demo guide and the API reference to make your custom firmware based on the non-OS SDK. For the RTOS SDK I see that the IDF is the way to go, following this guide.