LCD touchscreen possibility for reprap firmwares

I thought it might be good to use an lcd with touch screen for some of the sd card firmwares coming out. I've started working on optimizing an lcd driver, and hope people will start considering lcd touch displays for firmwares.

code available
[www.thingiverse.com]

youtube video

touch driver is sensitive enough that i made simple code to allow pinch zoom.

lcd still needs rotation, itoa that is faster, and library that makes it easier to know if a button created has been pushed,

driver works a bit faster for an 8bit mode display. it used the fastio reprap library, and some caching tricks to avoid redundant writes, also can scale font size.

enjoy, and hopefully think about using them....

some things that would be possible, is previewing a build, seeing an image of what is about to print, so you know the file is correct, more display options because buttons can be programmed in.

Wow! What an impressive thing for such a tiny microcontroller!

How much CPU cycles does this thing need? How much of these CPU cycles are required to be realtime? What happens when the ATmega is exhausted for a few seconds with other duties? These are the questions important for getting it work in a RepRap.

Is the assumption right it uses 8 I/O pins? I'm not sure wether Sanguinololu and/or Gen7 can spare that many, much less on a single port.

---

Generation 7 Electronics	Teacup Firmware	RepRap DIY

LCD uses 8 data pins and 4 other pins to control the lcd. but it works in 16 bit mode, 8bit mode and spi mode, spi would also reuse some pins because of addressing. I will look into pin count for that as well. Also i took care to make sure all writes to i/o were processed at the pin level. no specific port is necessary. just i/o pins and 2 pins that have a/d.

Lcd refresh rate and memory buffer are built into the LCD display driver. so do it once in cpu, and it stays on screen.

the most common update would be for text display for temp read outs, which actually takes about the same cpu cycles no matter what display used, time for number itoa to string, time for converting string to array buffer char and that is about it.

more complex work such as display setup, preview map, and images would more likely be done before the cpu is used for any move calculations.

also arduino is going to move to cortex processors soon enough, so cpu performance currently will not matter in the near future. but this should work fine with current arduino with pins to spare.

these type of displays also can work in a serial mode by spi. so in theory you could use grbl with lcd support even on an 328.

atmega1280 has pins to spare.

there is really not much more processing going on here than a regular char display lcd, what it adds is ability to do lines, drawing and images. those do take longer, but the arduino usually has 60% of its time to spare, just stuck in waiting loops. there may need to be some algorithm to allow proper timing, but Merlin firmware already has this. All that someone would need to do is modify code slightly for this type of display, and modify the pin/button checks for the a/d input of the touchscreen.
Also the lcd code can be interrupted and the data would still be intact for when returned.

I will get those hard numbers for cpu cycles to display text, cpu cycles to get a touch response. but it is something in the less than milliseconds range, touch may take ~4ms because of multiple a/d reads, and about .5ms if just polling/checking if it has been touched. there are lcds that have an interrupt pin when pressure is sensed.

I think this type of display can easily be Incorporated into future firmwares.

Edited 1 time(s). Last edit at 06/05/2012 12:29PM by jamesdanielv.

I like the idea of having a nice touch screen to control your printer. In fact, I already bought such a display for that purpose. The graphic library for that display seems quite slow, something like 0.5 seconds for a complete fill. From the video your code looked much faster, but I think it is still not fast enough to work inside the firmware. Having written my own firmware I know how less room for extra computations is present. With your 60% cpu usage you might be right for many cases, but there are also scenarios where the cpu performance causes a slow down. Especially if you have many short moves the cpu usage increases dramatically. Then there are ui tasks that take more power like scrolling through a menu. Don't forget the use might want to change something during print! An other factor could get ram usage. I don't know how much ram is needed for gui management. I understand the graphics is stored in the lcd but the gui needs to know where the buttons are etc.

I think the best way is to make a interface protocol for external controller and handle the lcd/touch screen on a separate controller. One easy way would be to add a second input device for rx1/tx1 and handle it the same way rx0/tx0 is controlled. Then you could write a mini host software for the lcd and even add a sd card for sending data. That way the interface will always respond fast and doesn't disturb printing.

---

Repetier-Software - the home of Repetier-Host (Windows, Linux and Mac OS X) and Repetier-Firmware.
Repetier-Server - the solution to control your printer from everywhere.
Visit us on Facebook and Twitter!

The memory is quite low.The resources required and cpu cycles will be the same as using a 4 line display currently.

many serial (not spi serial) displays only communicate at 9600baud, and require a lot of data be sent for each line.

here are time tests take by this method:

unsigned long timetest=micros();
//tested function
(here its the meat)

//end of function
timetest=micros()-timetest;
Serial.print("function name :");
Serial.println(timetest);


it takes 43ms to do a full screen refresh currently. that is just shy of 24 times a second refresh!
it takes to write a single pixel.32-44 us, depending on cached or uncashed.
to write 324 pixels burst mode 2.8ms

to write alphanumeric char to display largest font 18ms
smallest font 1.4ms

int number to str=60us(); //using itoa it will be 32us per conversion

touch speed with full sensitivity measure time:
~600us// not programed to yet, but can detect in as little as 10us

I just don't see how processor speed is All the issue, when other displays currently used are slower.

Quote

atmega1280 has pins to spare

That's true, but SMT chips are currently considered to be a no-go for do-it-yourselfable electronics. The ISP option looks good, though, just one pin for a device select and a number of already used pins.

Quote

I just don't see how processor speed is All the issue

I don't see this either if the graphics library can reasonably live with extended interruptions. The bottleneck is a fast movement shortly before reaching maximum speed. While maintaining maximum speed there is no need for acceleration calculations, so there is some time to spare.

---

Generation 7 Electronics	Teacup Firmware	RepRap DIY

For comparison, the slowest display I have working is a 4x16 with i2C connection. I can update in less then 10ms. I update the screen every second.

What I didn't register the first post is, that your display seems to be monochrome. Mine has 16 bit color depth, so updating this is much slower.

I do update the display in the main thread, so it doesn't disturb any stepper handling. It only delays the refill of the printing buffer and temperature updates. As I already said, for normal prints this should be no real problem. Only in small circles with many segments you get timing problems - which you also have without displays. The analog reading of the touch position would be done in the same interrupt that reads the adc values for the temperatures, so using the results is just reading the last stored value and very fast.

So it would work inside the firmware. The color displays will not work good (16x slower). The biggest problem is making a nice gui from it. Or use it like the simple character displays and add touch functionality to replace the keys. You could use the extra space for some buttons and show the normal character output to the left. With your smaller font size this should be no problem and it makes updating faster, because only the left side needs updating. And it makes navigation easier. It would be hard to hit the right row with the small font size. With bigger font sizes you can't show as much information and would lose some of the advantages.

---

Repetier-Software - the home of Repetier-Host (Windows, Linux and Mac OS X) and Repetier-Firmware.
Repetier-Server - the solution to control your printer from everywhere.
Visit us on Facebook and Twitter!

thanks but the display is 262,000 color 18bitmax. it is running in 8bit mode.

as you scratch your head, let me just tell you all the wasted rewrites the normal drivers send.

i put great effort into optimization, the best thing people can do is try to have firmware use this type of display.

this driver is only 50% of the current potential.


colors defined for the user are red,green,blue,black,white,grey,yellow,cyan, purple, it does images as well, although i have not written code in to do so yet

just try the code. i worked hard optimizing it, configured the touch driver to be super sensitive, and added my own touch which is pinch zoom, i don't thin that existed on resistive displays for arduino before my version of driver.

I will continue to optimize the display code, what i hope for is 3 or four people to work on firmware. anyone interested in a particular firmware to work with this lcd, send me a personal message from this form, let me know your e xperience with reprap firmware, and i will pay for and have shipped to you one of these displays? Why because this is a possible future for reprap.

i will send out 1 display to
Repetier firmware
teacup firmware
Sprinter firmware
marlin firmware.
I prefer to send to the most experienced person for each firmware, hence the inventor, major contributor, but will send displays to the most qualified applicant for each. my only requirements are that the display works using touch panel to control software, and that the design be based for an atmega. My code is open source, the only thing i am not a 100% sure of is the type of license MIT has for touch library. I Will find out.

40$ a piece, out of my pocket after shipping, sent out within a few weeks. why really i want to use the best of the firmwares for my own printer, and This allows me to focus time on finishing and optimizing this lcd color driver.

It does not matter the manufacture or the pin out. most lcd touch displays work the same. the expensive ones work faster.

Edited 3 time(s). Last edit at 06/06/2012 06:29AM by jamesdanielv.

When you get this timings with a colored display I really have to have a look at it. Must be some good optimizations. The display I have is this:

https://github.com/watterott/MI0283QT-Adapter

the display is controlled via spi. Is this also possible with your library?

BTW: What display do you use and which displays will work with the library? If it is possible to use it with different display/driver types it is even more interesting.

I'm very interested in trying how good it would work with my firmware. Writing it like I described above should be fairly easy, as the ui code already supports the extension with different display drivers. The only thing is, I have some other things to do first, so I could write the code somewhere in july and then include it in the official branch.

---

Repetier-Software - the home of Repetier-Host (Windows, Linux and Mac OS X) and Repetier-Firmware.
Repetier-Server - the solution to control your printer from everywhere.
Visit us on Facebook and Twitter!

i will look into spi later on, but if you send me your address name and firmware you will work on, i will purchase and mail out the display model have specifically used to you. This is a personal project. i won't give your address or name out, but i need it to address the mail package too you. The supplier i get these types of displays from is in honk kong, it takes about 3-4 weeks for items to arrive.


there are 3 types of displays common for microcontrollers.
here are the data sheets
ILI9325 [www.adafruit.com] has spi or serial ability
HX8347 [www.displayfuture.com] has spi ability or serial
ST7781 [www.wide.hk] [www.soiseek.com] says it is spi capable


what is interesting here is each command is 2-3bytes, each command is followed by a 2 or 3 byte instruction.
in 16 bit mode each command is 2 bytes, and each instruction is 2bytes.


typical drivers do this

command-instruction xpos (2+2)
command-instruction ypos (2+2)

command-instruction color of pixel (2+2)
that is 12 bytes for one pixel.

but the display remembers its x,y, and color, and each time you don't specify x or y, the display automatically advances by 1 just by sending the color information

so in most cases, just send color information, with caching techniques it is possible to know when an axis needs updating, and when color changes.

for color, if using 8 bit color, where f8f8 (same byte twice) it is possible to just leave the data the same, and toggle wr high and low.so essentially performing an update with out changing any bytes. there will be some minor differences for spi, but spi can use the arduino buffer to perform tasks independently, meaning while spi may be slower, it is faster because it processed data twice as fast or more, spi is done in the background with hardware assistance.

Edited 3 time(s). Last edit at 06/06/2012 10:43AM by jamesdanielv.

You can check this out, [shieldlist.org], is pretty much fast, I think.You'll just use 2 pins and you leave the the the core process loosed.
Or you can check every kind of shield there are available currently,here: [shieldlist.org]

bye

there currently are some issues with using spi, spi forces the entire data line to be written 1 bit at a type, there are a few solutions i would like to discuss. Also i am open to any other solution as well.

the goal is for high performance graphics and text with lcd touch screen. it currently works with 12 pins available. entire screen can be changed in less than 39ms now.

it can work as is if pins are available
it can also work with spi, however slower, maybe 2-4 times a second for screen blanking
it can work with a chip translator very quickly using spi to talk with another chip , then the lcd would be managed by this other chip.

or using a shift register for the 8bits and 5 free pins. most of the time saving is writing similar 8bit data, so loading the shift register even 1 bit at a time does not slow down the display as much as spi would.

any options or ideas, any way to get spi faster than 8mhz on arduino?


its not a dead end, it would work great for those with atmega 1280 boards to hook up directly to lcd.

Ok, I've already given out a display to repetier , and as per his suggestion he wanted high speed square fill boxes, I am still working on changes to display touch driver, and to the line draw feature, but thought i would post what i have currently on thingiverse. here is a video of what it has so far which basically is just the added square fill, and the touch now knows when it has been pushed and held. also doubled char display speed, and increased screen refresh to 25 times a second
[www.thingiverse.com]

I also know know that MIT licenses are considered ok for general use, just there are some requirements for disclosing all the files along with the MIT files. not a big deal, but I still plan to change the display touch driver to be more compatible with reprap firmware. also it will know if an area is pushed, expanded, double tapped.

still working on serial mode using spi. it seems using a shift register will be faster than actual spi. so i will be designing a board for that and posting it on reprap as soon as it works, and as soon as it is tested.

By the way, again I want to state i have purchased 4 of these displays just to give away to people that work on firmware. I prefer people that have arduino MEga, but if not there will be a way to connect it in spi, and also with shift register spi. the chipset used for this lcd is ili9328. just send a personal message with name and address and what experience you have with firmware.

here is a video of recent work.

Edited 3 time(s). Last edit at 06/20/2012 11:24AM by jamesdanielv.

ok, updated again, this update cleans up lcd code and places it into lcdDriver.h, rest can be executed from anywhere. working on the same for touch driver, version is now 0_3 also just add lcdDriver.h to your code and you have lcd control. touch is going to be done similar.

after touch code minimized and modified will also provide simple sample code for marlin firmware.

still needs
spi code, different init for different displays. code still can have any pinout setup for lcd shields.

code is here
[www.thingiverse.com]

commands available for display so far are
lcd_Clear()
lcd_Rectangle()
lcd_Rectangle_Fill()
lcd_Circle()
lcd_Str()
Init()

next version will contain code for shift register spi which will reduce pins needed, and include notes for changes with version

Great work, jamesdanielv. To be honest, I'm wrestling with myself wether I should take your offer to fit the code into Teacup. But then I have Gen7 electronics only (a whole bunch of them), which pretty much requires attaching a display via SPI, as 12 or 14 pins simply exhaust what's available on through-hole ATmegas. And, as always, there's a big stack of stuff I'd like to get done on other projects. SD card for Gen7, look-ahead for Teacup, you name it.

Now I'm looking forward for the SPI connection ...

---

Generation 7 Electronics	Teacup Firmware	RepRap DIY

well it is almost completed. the display part is done, just need to work out a few things for touch functions. For display how is 1 spi cable, and 1 select pin. also what do you guys think of the speed response? i will post a schematic of how it is hooked up. I battled with all sets of thoughts on using special chips, and settled on the 74ls164 shift register. it is basic and will likely never be in short supply, can be switched with a different shift register with little or no modification , and it is easy for everyone to use and implement since it requires no programmer. less than a buck cost including diodes, and 1 capacitor. I'm trying to design the circuit board traces so that it can be traced with a reprap printer, then etched. this version uses hardware spi.

Edited 1 time(s). Last edit at 07/14/2012 06:57AM by jamesdanielv.

Excellent!

Now, if you could post the schematics of this daughter board, I'd like to make an extension board for Gen7 making this display possible. Gen7's extension board provides 3.3V already.

Didn't I read somewhere you also need two analog pins for the "touch" part of this display?

---

Generation 7 Electronics	Teacup Firmware	RepRap DIY

Sorry - Kind of hijacking the thread...

I highly appreciate the effort you have put into this. 7 years ago when there was no iPhone I made a touch-screen remote control for my course project. While char-based LCD and graphical based LCD are kind of standard, I wouldn't know where to buy the touch screen. I was using a Palm replacement screen in my project.

To me, maybe it's even more fancy and cost effective if the control could be done from Android/iPhone, through bluetooth or Wifi.

thanks. touch screens are quite common in surplus and purchased on the following sites for example

[wide.hk]
[adafruit.com]
[www.mouser.com]
[www.nuelectronics.com]
it is another option that is out on the table. time have changed, tech is everywhere now.

Traumflug:

"Didn't I read somewhere you also need two analog pins for the "touch" part of this display?"



yes 2 adc's are currently needed. 4 pins are required for the touch panel but 2 pins are tied to the shift register. I'm trying to get it to use 1 pin adc but have not figured it out yet. either way expect a circuit diagram as soon as i verify the actual board and layout works.

alright, I'm getting close. here is the parts list for those that are interested.

(1) 74164
7 diodes
2 200ohm resistors
3 100ohm resistors
4 .01uf capacitors

i have two ways to use the spi, and i am finishing up on a method to take advantage of the 8mhz clock to do large writes to display. arduino i/o can only do 2.5mhz, but spi clock is 8mhz. if it works and i can get the clock to work independently then it will make the display between 2 to 4 times quicker. there are issues with trace cross talk I'm addressing, along with noise suppression code for the touch screen for reliable operation in noisy environments. MIT driver has been replaced with my own code that will be open source.

I will post schematic here in a short time, I'm not quite sure when but it will likely be within the next week

Also will post updated code that takes advantage of the spi interface.


There was considerable thought in this design. it will allow for duplex operation if the sd card is plugged into the spare spi port on the daughter board, so that the display can be written to while the sd card is being read for example, maybe not the next release, but in a future release. it is at least possible.


required pins for i/o are the spi port, and 3 i/o pins, 2 of which need to be adc for touch functionality

I have started a wiki page here [www.reprap.org] and here is the draft schematic. also the code should be posted on [www.thingiverse.com]. It will be in git as well shortly. This is not the final version, but it is a version that works well. it is supported by code. here is a direct link [www.thingiverse.com]

here is a simple schematic.



SPI method using shift register. Version 1 draft

This spi version allows sending as little as 1 byte of data to change pixel data on display. other spi methods require 10 bytes or more. lcd is running in 8bit mode, meaning transfer of 8 bits of data at a time, it allows for up to 16bit color.


It falls within the loose spec of spi.
Spi is enabled and disabled in registers after each byte is loaded onto shift register. Shift register acts as a buffer.
1 wr is also the select pin. If wr is set high and not brought low, then the display does not care what the shift register data has. This pin is normally toggled.

If this is the device in communication, then after spi sends byte data, spi can be disabled and MISO pin can be set as an output for (rs) reducing pin count. Software sets it back to input when spi is re enabled. if register enable/disable of spi it takes minimal time. currently most of the i/o speed comes from buffering of 8 bits of data, and fast toggle of wr pin. this design works and is what is bread boarded. at this point it time i would not commit to permanent board design until a final board layout is posted. Also the 2 adc pin touch is still buggy.


for those interested. the alternate version ties wr sck when high, and when low shift register is tied to sck.

Edited 7 time(s). Last edit at 08/21/2012 07:16PM by jamesdanielv.

Looks pretty simple. That's good!

---

Generation 7 Electronics	Teacup Firmware	RepRap DIY

It has been a while since I last updated. I'm thinking about adding in 2 more discreet logic chips. Another 74164, and a 4066 (a 4 channel analog switch) this allows the ADC to be directly controlled by the board for the ADC signal to the microcontroller. It also allows switching to be on the daughter board as well. This addresses an issue with selecting ADC and sampling as now only 1 channel needs to be monitored, and it can be directly controlled by the LCD driver code. It just seems better to use discrete components. I will post the updated schematic in a week or so, and then finalize the daughter board. Things are moving more slowly than I intended, but I think I have a vendor convinced to manufacture the daughter board as well. Time will tell. Schematic data will be updated in reprap wiki.

The daugher board will handle sd/lcd/touch spi inputs, and manage the clk for sd so sd can recieve data at same time the lcd is recieving data.

I don't think the lcd driver can have control over the adc. In Repetier and Marlin the ADC is checked in a interrupt loop, external adc requests would destroy the temperature reading. If I understand you right you need 2 analog signals. Now you want to add a switch somehow that changes the analog input signal. Changing the analog signal needs another pin for selecton I guess? Thats making it extra complicated and also would reduce the polling frequency, at least in Repetier. What is the benefit trading a analog pin for a digital one? Might be missing something, but that is how it looks to me.

---

Repetier-Software - the home of Repetier-Host (Windows, Linux and Mac OS X) and Repetier-Firmware.
Repetier-Server - the solution to control your printer from everywhere.
Visit us on Facebook and Twitter!

it is probably best the daughter board perform its own adc anyway, with restrictions to a single layer pcb that can be reprap made and mass produced at the same time, complexities arise with noise. jumper wires acting as antennas and wide traces allow for cross talk.


you can control the adc thru the spi. also a single channel adc spi adc converter MCP3201-CI/P-ND is 2.70 in single quantities. it does 12bit adc and 100k per second. no reliance on the processor for adc conversion.

the reason for 1adc wire was to reduce noise and cross talk.

originally i had intended just to have it work with atmega 1280 and bigger processors, but there seems to be interest in anything that reduces pin count and runs at high speed. the original code will still work, just the spi code will migrate towards the multichip solution.

Edited 2 time(s). Last edit at 10/24/2012 05:49AM by jamesdanielv.

That should give you full control for the driver:-)

One more thought. The display itself has pins to connect to a arduino. I think you will make a doughter board where the display is put on. Could you add a sd card connector to the left side of the board like with the popular reprapdiscount lcd controler. With your lcd you block the spi pins and that is the only other device using the spi for most printers. That way you have only one connector for all spi driven ui devices.

---

Repetier-Software - the home of Repetier-Host (Windows, Linux and Mac OS X) and Repetier-Firmware.
Repetier-Server - the solution to control your printer from everywhere.
Visit us on Facebook and Twitter!

yeah, i know we would go there

following