Jump to content


ebuild is sad to announce its closure - it has become too time and resource intensive to develop, manage and maintain.

However, ebuild will remain on-line in archive mode (ie no posting facilties) for several weeks so that users can use it as an information resource.

Esp8266 - Roll-Your-Own Internet Of Things


  • Please log in to reply
16 replies to this topic

#1 TerryE

TerryE

    Regular Member

  • Member Blogger
  • PipPipPip
  • 709 posts
  • LocationSouth Northants

Posted 23 March 2015 - 01:06 PM

I've raised the ESP8266 on my Climate Control (Home Automation) For My New Build topic, but didn't think that I should go into details on this forum. However Neil suggested in #79 that this might be of interest to other members and would be a good topic for Boffin's corner, so here goes. First the big (or more accurately little) reveal:
Attached File  esp8266-01-thermometer.jpg   29.28K   17 downloads
The foreground shows my first iteration design for a remote thermometer. You can see the WiFi aerial on the top daughter-board. It is powered by the plug to its right and my Sony USB-stick MP3 player behind is shown to give you an idea of scale. The whole build cost is less than a tenner. Working from right to left:
  • USB power connector. This is a bit of a botch cannibalised from a USB cable, since the proper part is still on its way from China (it's also on the wrong side -- the WiFi aerial should be facing out from the wall.)
  • USB power connector header. Not sure why I put this on.
  • 5v to 3.3V step-down daughter board. The USB plug produces 5V but the electronics run at 3.3V. (Behind this is a jumper to put the board into Firmware flash mode, but I will probably drop this, as its just easier to plug the processor into a dedicated flasher board.
  • Header for the ESP 8266 ESP-01 chip (with processor in place). This is the processor, memory and WiFi on a single tiny daughter board.
  • 4 x one-wire headers. This is where I plug in my remote digital thermometer (DS18B20) chips. A simpler version of this board would just have one mounted thermometer.
  • 3.3V serial header. This is used to flash the firmware or when programming the chip. I use a standard USB-to-serial chip to connect to this from my laptop. It's not connected for normal running.
The whole thing is on a 2x8cm through-whole prototyping board. This is the first time that I've used one of these (and over 20 years since I did any electronics soldering). Using this has been a bit of a learning curve. I picked up the techniques as I went along, so the next one will be a lot quicker and better. The board is mainly used to route connections to the daughter boards and connectors, though there are a couple of drop-down resistors and some variants recommend a decent capacitor between the 3.3V and GND rails, and which I might add.

This version supports firmware flashing, SW development over USB as well as 4 thermometers. I might want one or two for a "production" version, but I decided to put in headers for four as I want to get a handle on how accurate they are.

Out of the box, this ESP-01 board comes with firmware to provide an modem-style AT interface over TTL serial so that you can use it with an Arduino or the the likes. However, I have reflashed it with a Lua environment, developed by NodeMCU, and which is basically a straight port of the eLua environment to the microprocessor which is embedded in the main chip on the board. This enable me to program the board in an efficient late-bound (like Python) language, but with cut-down more C-like syntax. It uses standard modules (coded in C) to do all of the grunt work. These include pretty much everything that you'd want for a device like this:
  • External hardware interfacing: ADC, I²C, one-wire, SPI, PWM, UART, WS2812, U8G graphics
  • Communications: Net, Wifi, MQTT, CoAP, CJSON
  • Other low-level support: GPIO, BIT manipulation, File I/O, Timer
So as well as being able to act as a web client and a web server, the device also supports MQTT and CoAP which are the current "best-in-class" for secure and efficient device connection over TCP. Pretty amazing for something costing a couple of £.

My previous "project" was working on the internals of the PHP (the language that most of the web is written in) compiler and runtime system, so I am used to crawling around inside this sort of software. The tl;dr summary is that I am impressed, and the deeper that I dig I continue to get more impressed. I could go on, but I pause for other comments.

Edited by TerryE, 23 March 2015 - 01:27 PM.


#2 temp

temp

    Advanced Member

  • Moderators
  • 10,200 posts

Posted 24 March 2015 - 09:36 AM

If you plan to make several it might be worth looking at some of the cheap PCB fabrication services that exist these days. For example an 80mm * 20mm double sided plated through hole PCB with solder resist both sides... can get three PCBs for £75 plus postage making them £25 each from... http://www.pcbtrain.co.uk/ and that was just the first one I looked at so might be cheaper out there. Unit price falls rapidly with quantity. Could also be a lot cheaper if you can do it on a single sided board.

If you want ultra cheap Google found..

http://hackaday.com/...-cheap-pcb-fab/

Quote

The boards are definitely cheap. $12 USD gets you ten 5 cm by 5 cm boards with 100% e-test and free worldwide shipping. You can even choose from a number of solder mask colors for no additional cost. [Ian] does warn the boards aren’t of the best quality, as you can tell in the Bus Pirate picture above. The silkscreen alignment has some issues, but for $1.2 a board, it’s hard to complain. After all, the site’s motto is “No bull, just crappy PCBs.”




#3 Alphonsox

Alphonsox

    Advanced Member

  • Members
  • PipPipPipPip
  • 1,495 posts
  • LocationCounty Down, NI and Forest of Dean, England

Posted 24 March 2015 - 11:58 AM

Looks interesting - What is the power requirements for the board ? Is a battery based solution possible ? I guess not with a full scale wifi on board.

#4 TerryE

TerryE

    Regular Member

  • Member Blogger
  • PipPipPip
  • 709 posts
  • LocationSouth Northants

Posted 24 March 2015 - 01:00 PM

@Colin, I thought about using a custom PCB, and out of interest even looked at doing my own. (It's quite an easy process if you have a laser printer, but you have to use a daughter board for the esp8266). A friend of mine makes some specialist measuring instruments and sells 10s off per year and goes this route. However ESP already do 12 variants of their chipset boards with different pinout/surface mount configs; different I/O connectivity off board; external aerial connection; and different SRAM sizes.) The 5V to 3.3V step down can be done with 6-8 discrete components costing £2-4 in 1-10 off volumes or on a small daughter board with 4 pinouts costing £2-3. So my "mother board" is really just some connection and bussing, plus somewhere to mount headers. For my planned volumes (1-5 units of any given config), it's easier just to use strip board or the through-hole prototyping board approach.

@Neil, when transmitting, the power draw is maybe 200mW, but when sleeping the power draw is minimal (a few mW), but still enough to drain a battery quite quickly if intended to be offering its service 24x7, hence needing a power supply, and why I decided on the USB stick format to permit the use of a cheap 5V USB plug. With a decent enclosure it isn't any more obtrusive than one of these plugin night lights, and I did consider putting a decent LED on this show you could even dual-purpose a room sensor for this.

There are now decent enough charging circuit daughter boards to allow you to use a small Li or NiMH battery and PV panel to allow you to have an outside (wall mounted, say) unit about 5x10cm which could run 24x7 and still pump out readings every min or so.

However, the main thing about this eLua approach is the ease with which you can customise the onboard logic to the particular use. This allows you to use TCP over Wifi running a decent truly secure applications stack based on the open CoAP standard, rather than the obsolescent LightwaveRF or Z-wave stacks (whose main security mechanism are ignorance and the non-open source base, both of which can be trivially hacked).

Edited by TerryE, 24 March 2015 - 02:01 PM.


#5 recoveringacademic

recoveringacademic

    Advanced Member

  • Moderators
  • 1,343 posts
  • LocationLancaster, Lancashire

Posted 24 March 2015 - 02:53 PM

Terry, I like to start my thinking with an end-in-mind.

Help me here - at the end of a successful project, what would a JQuery, CSS3, HTML5 coder be able to do with a 'thing' you built?

#6 DamonHD

DamonHD

    Advanced Member

  • Members
  • PipPipPipPip
  • 1,046 posts
  • LocationLondon, UK

Posted 24 March 2015 - 06:30 PM

View Posttemp, on 24 March 2015 - 09:36 AM, said:

If you plan to make several it might be worth looking at some of the cheap PCB fabrication services that exist these days. For example an 80mm * 20mm double sided plated through hole PCB with solder resist both sides... can get three PCBs for £75 plus postage making them £25 each from... http://www.pcbtrain.co.uk/ and that was just the first one I looked at so might be cheaper out there. Unit price falls rapidly with quantity. Could also be a lot cheaper if you can do it on a single sided board.

If you want ultra cheap Google found..

http://hackaday.com/...-cheap-pcb-fab/

You can do better than £1/board with my favourite iTead:

http://imall.iteadst...m120418001.html

Rgds

Damon

#7 Alphonsox

Alphonsox

    Advanced Member

  • Members
  • PipPipPipPip
  • 1,495 posts
  • LocationCounty Down, NI and Forest of Dean, England

Posted 24 March 2015 - 07:19 PM

View PostDamonHD, on 24 March 2015 - 06:30 PM, said:

You can do better than £1/board with my favourite iTead:

http://imall.iteadst...m120418001.html

Rgds

Damon

That is very, very cheap - Thanks for the link

#8 TerryE

TerryE

    Regular Member

  • Member Blogger
  • PipPipPip
  • 709 posts
  • LocationSouth Northants

Posted 24 March 2015 - 10:16 PM

@Ian, if you think about how you would interact with your house in an automated world:
  • you <-> your PC / tablet <-> some server <-> the actual bits that need to be controlled
JQuery, CSS3, HTML5 are all technology sets which allow the personal device developer (eg. for you PC or tablet) to create an App which gives you a nice experience.

However it will still need to connect to some system which provide a Home Automation (HA) service which is customised to your house and your preferences -- e.g. which embed the rules for how you want your house to run by default if you don't say otherwise -- and which talks to the actual bits of hardware, the devices in your house. This is where applications like Loxone, OpenHAB, Domotizc, Pimatic all sit.

However, the home automation system itself then needs to connect to the real world: the devices in your house and the sensors which it uses to collect information to inform its control. And this is where CoAP and my little H/W module come in. The HA system wants to "ask": what's the temperature of the TS and the input and output feeds; or tell the window blinds in the front bedroom to close. It wants to treat these as a "subroutine call" over some amorphous network. This little bit of hardware and its eLua firmware allows you to build a thermometer, relay, whatever into a little callable brick -- in a secure and open way, and for a few £.

OK, you don't have the memory or processing capacity on the ESP8266 to do an awful lot, but that's OK because one chip will only typically do one thing.

Personally I am a little paranoid about security, so I hate the idea of any Android/iOS phone or tablet being able to connect to my house's device's. Only my HA system should be able to do this, and only properly authenticated end-user devices should be able to connect to it.

Edited by TerryE, 24 March 2015 - 10:19 PM.


#9 temp

temp

    Advanced Member

  • Moderators
  • 10,200 posts

Posted 25 March 2015 - 09:01 AM

View PostDamonHD, on 24 March 2015 - 06:30 PM, said:



You can do better than £1/board with my favourite iTead:

http://imall.iteadst...m120418001.html

Rgds

Damon

Woe that's cheap. Thanks for the link.

#10 recoveringacademic

recoveringacademic

    Advanced Member

  • Moderators
  • 1,343 posts
  • LocationLancaster, Lancashire

Posted 25 March 2015 - 09:22 AM

Thanks Terry. I don't need any more excuse to get into Lua - I'm planning to start working on writing some little routines for model aircraft / quadcopters with video cameras attached (roof inspection?).

I'm really very interested indeed to watch the progress of your project even though I have to go for a percentage in terms of understanding. The more often I re-read your posts the more the mist seems to rise.

Yes, I agree with you about phone o/s, JQ et al. - I was thinking more of the fun I could have with coding challenges to slow down my descent into being hard(er)-of-thinking.

Kind regards. Ian

#11 temp

temp

    Advanced Member

  • Moderators
  • 10,200 posts

Posted 25 March 2015 - 10:31 AM

View Postrecoveringacademic, on 25 March 2015 - 09:22 AM, said:

Thanks Terry. I don't need any more excuse to get into Lua - I'm planning to start working on writing some little routines for model aircraft / quadcopters with video cameras attached (roof inspection?).

The CAA are getting a bit twitchy about that now...

http://www.caa.co.uk...95&pageid=16006

Quote

You must request permission from the CAA if you plan to:

•fly the aircraft on a commercial basis (i.e. conducting ‘aerial work’)
or
•fly a camera/surveillance fitted aircraft within congested areas or closer (than the distances listed within Article 167) to people or properties (vehicles, vessels or structures) that are not under your control

Article 167 precludes flying within 150m of a congested area or 50m from a structure (aka building) not in your control (eg neighbours of the house being inspected).

Thousands of people who got them for Christmas are probably breaking that last rule by flying in their own garden.


Edited by temp, 25 March 2015 - 10:33 AM.


#12 recoveringacademic

recoveringacademic

    Advanced Member

  • Moderators
  • 1,343 posts
  • LocationLancaster, Lancashire

Posted 25 March 2015 - 11:08 AM

View Posttemp, on 25 March 2015 - 10:31 AM, said:

Thousands of people who got them for Christmas are probably breaking that last rule by flying in their own garden.

Yes, exactly. I am an experienced qualified pilot, and qualified model pilot. I am also fully insured (a few million).

I am working in this area precisely because the sport needs a disciplined approach: more, it needs people to demonstrate that model flying can be done safely, and well - and be of use to the local community.

Several of us are in the early stages of planning a program which gives early warning to the authorities about people stranded on Morecambe Bay (my back-yard) for example. Thirty square miles which are totally safe in model flying terms , and a rather useful -if amateur- service.

#13 SteamyTea

SteamyTea

    Advanced Member

  • Members
  • PipPipPipPip
  • 4,322 posts
  • LocationCornwall

Posted 26 March 2015 - 08:05 AM

Strange how we want restrictions on some technologies, but hate them when it comes to planning.

Now a serious question or more:
How do you configure all this kit? It is easy to cobble together hardware, but then getting it to talk to things is hard I find (I still like spanners and screwdrivers).
How easy is it to get this remote stuff to switch things on and off, like a storage heater, water heater.
How easy would it be to add a solar sensor that reads W/m2 to it?

I like the cost and size, just got no idea how to make it all work together.

As for security, can't something be done with Tor. It is a simple idea that seems to work well, I use the Tor browser and have a Tor node. Really can't understand why it is not used more for security sensitive things.

#14 TerryE

TerryE

    Regular Member

  • Member Blogger
  • PipPipPip
  • 709 posts
  • LocationSouth Northants

Posted 26 March 2015 - 03:46 PM

If anyone is uncomfortable with building their own hardware, then stick with Z-wave or LightwaveRF.

I am still in play mode at the 'mo. I have my little stick with 4 digital thermometers pushing up a set of readings every 5 mins to my RPi over the Home Wifi. If I wanted to make this stuff publicly accessible, then I would push them up to my ellisons.org.uk service and use this as a portal.

As to the how easy Qs: watch this space.

The threat here is that some idiot starts hacking into your system and playing god with your lights, etc. So you want to have a private non-spoofable transport between your devices, your server, and user interface devices. What this MQTT and CoAP crap does in effect is to expose these embedded devices though a remote procedure call (RPC) interface to a properly authorise server (your HA system).

ToR as the design was sponsored and funded by the NSA was so that dissidents in China, etc. would be able to share their thoughts and secrets with the western world. Pity for them that they forget that the Western powers seem to be as big villains in the international politics world and that their own citizens are now resorting to these technologies to avoid being spied on by their own as well as other governments, and criminal ditto. I am not worried about ELINT threats for HA, so ToR is total overkill.

I personally wouldn't open any ports on my home LAN for any internet-wide client access. If I wanted internet-wide access then I'd either use an OpenVPN solution to allow my tablet on roam-about to tunnel securely into my home network or simpler, just use a portal on my public presence as I mentioned above.

Edited by TerryE, 26 March 2015 - 03:48 PM.


#15 TerryE

TerryE

    Regular Member

  • Member Blogger
  • PipPipPip
  • 709 posts
  • LocationSouth Northants

Posted 29 March 2015 - 08:44 PM

Durrrhhh, still having fun. My little device is working well enough and I am spending a lot of time getting to know the internals of the eLua runtime system -- because that's my sort of hobby speciality (when I am not planning house-builds). I did make a couple of mistakes in my prototype design though, mainly that it's too close to a final config and hasn't been laid out with development testing, e.g.:
  • It's missing a hard reset switch (you do this a lot when testing)
  • It's missing a firmware flash switch. You do this less, but it's still an occasional necessity.
  • I should have mounted it on a 3xAA battery pack -- just to get one less power cable out of the way.
  • The UART is a 6 pin connector to a little external board. I should have just soldered one (a USB to UART daughter board) onto a slightly larger prototyping board to keep the clutter down when developing.
  • I should have taken all of the I/O pins out on a header for breadboarding.
Live and learn :)

Edited by TerryE, 29 March 2015 - 08:46 PM.


#16 TerryE

TerryE

    Regular Member

  • Member Blogger
  • PipPipPip
  • 709 posts
  • LocationSouth Northants

Posted 13 April 2015 - 11:13 AM

Attached File  mark-II-esp8266.jpg   89.44K   7 downloads

Well here is Mk II -- it's really my dev board as a I discussed in my last post.
  • I've taken off all of the thermometer headers and added a breakout header so I can do any prototyping on a proper breadboard next to it (though I have left some space on the board to add stuff if I need to).
  • The FTDI232 chip plugs directly into the board on the deck.
  • I've added a 220µF cap to stabilise VCC to GND
  • I can simply jumper a couple of pins on the breakout header with a breadboard jumper to do a H/W reset or to go into flash programming mode.
  • I didn't bother with the battery option.
The soldering isn't brilliant, because this was the first time that I've worked with using wire-rap wire on this type of through-hole board and there is a definite technique to it. The wire is single core and can fatigue quite easily so you really need to solder both ends at the same time. (The two bare wires are VCC and GND buses; it's just easier for these to be exposed if you want to add extra components.) Still the board works brilliantly, and the Lua system is nice.

This approach of using a through-hole board with 40 SWG wiring is very good for this sort of 1 to 3 off prototype development, and also permanent enough to be used for home use.

I am posting on the ESP8266 forum and I'll put my code and documentation on this and how to use the Lua environment in general in Github when I get around to it. I'll post any relevant links back here.

PS. The reason for the wonky ESP8266 daughter board is that my header strip has the pins slightly offset so I should have soldered the 2 off 4x1 headers in the same orientation. I didn't so the two rows are slightly skewed w.r.t. each other (50:50 chance if you don't realise this) -- and hence the daughter board skew -- bugger

Edited by TerryE, 13 April 2015 - 11:20 AM.


#17 TerryE

TerryE

    Regular Member

  • Member Blogger
  • PipPipPip
  • 709 posts
  • LocationSouth Northants

Posted 26 December 2015 - 01:14 AM

I am now one of the core developers of the Lua firmware build for the ESP8266s.

Lua is a fairly simply high level language that is easy to get to grips with for basic sequencing if you have any programming experience. But make sure that you buy ESP modules with at least 1Mb Flash. The ESP-01s with 1Mb cost about £2 each from Bangood. These have limited GPIOs exposed but are fine for a single relay, a set of onewire sensors, or one I2C bus. The 12Es and 201s have a lot more I/O pins exposed, but my approach is one device (or a few) per IoT chip, and connect them back to my Home Automaton S/W running on an RPI2 using a standard Rasbian server build. Using a simple bi-directional UDP protocol is just so much easier than bit-banging out over an RF link.

The HA system handles the user interface.

A special offer for any member of this forum: if you want support advice on using the ESP8266 (and especially if you are using the Lua builds) and you have any issues that you can't sort out via esp266.com or StackOverflow, then just PM me and we can swap emails / have a chat.

Edited by TerryE, 26 December 2015 - 01:20 AM.