Is there a way to automate editing of gcode files?

I'm not sure if this is the place to post this but I can't find a better place on these forums.

Basically, I have a Diamond Hot end and I'm playing around with mixing ratios. There are potentially hundreds, if not thousand of possible combinations of mixing 3 filaments in (say) 1% increments. I have created a 30 "colour" object by creating 30 mixing ratios of 10% of each of the 3 filaments (e.g 0.9:0.1:0.0, 0.6:0.3:0.1, 0.4:0.1:0.5 etc etc). Then, I created a 90mm high object and sliced it as normal. Then using notepad++ did a "find next" for "Z" on the gcode file repeatedly for every 3 mm and inserted the tool (mix ratio) command. Obviously this is a bit of a painstaking process. So I was wondering if there is a way of automating it.

I was thinking on the lines of maybe a text editor with some sort of macro function which would work something like this:

for j = (start, step,end) // colour A
for k = (start, step, end) // colour B
for l = (start,step,end) // colour C
Find next "m" in the file
insert a new line containing M567 P4 Ej:k,l // set mix ratio for tool 4 to the values of j,k and l
next l
next k
next j

I appreciate that the above would not work as the mix ratio always needs to add up to 1 but that could be solved with some extra maths.

So, is there any way of doing the above or something like that? I'd like to end up with a gcode file which would result in an object which had a smooth transition from one colour to the next over number of layers.

Both Cura and Slic3r allow you to write a script that would be run on the output of the respective slicer as part of the flow. Your script would have to search the slicer generated Gcode for whatever criteria you want, add whatever change you need and output the modified Gcode. Doing it this way you don't need anything else besides the slicer and your custom script.

Look up sed, the standard Unix stream editor. There are versions available for other platforms including Windows. I used it to change the tool numbers in gcode files a while ago.

---

Large delta printer [miscsolutions.wordpress.com], E3D tool changer, Robotdigg SCARA printer, Crane Quad and Ormerod

Disclosure: I design Duet electronics and work on RepRapFirmware, [duet3d.com].

Not a complete answer, but a step in the right direction?

In NotePad++ (and other editors), you can use Regular Expressions for Search & Replace operations. Using this, you can (e.g.) automatically insert something after every matching Z command.

e.g. To change colour every 10mm vertically, search for "([^\n*]Z(\d)0[^\n]*\n)" and replace with "$1M567 P4 E$2\n" would insert an M567 P4 E[x] line after every line containing a Z coordinate which is a multiple of 10, where [x] would be the tens digit of the height. e.g.

G0 X56.102 Y61.759 Z10.100

would become

G0 X56.102 Y61.759 Z10.100
M567 P4 E1

You would then do another pass through searching for "M567 P4 E" and replace the last digit with the mix value (e.g. 01:0.2:0.7) for that height.

I was curious about these post-processing scripts. I couldn't find anything about how it works in Cura, but in Slic3r you can run the script in any interpreted language (Perl, Python, Ruby, bash, etc). The way the Slic3r manual reads, this is designed around Unix command processing, (#! processor as the first line of the script file), but I would hope that the Windows association-by-filename-extension mechanism would work as well.

Edit:
I had a bit of a play with Perl handling a Gcode file as an example. Here's the result. It's not exactly what the OP wanted, but similar, and will provide a basis for more development... If the 2 specified colours A & B are >= 1 at any height, then they are mixed in their proportions. If they total < 1, then colour C is used to make the total up to 1. So if colours A, B, C are black,white,red respectively it starts out black at the bottom, fading into light gray over 100mm, at the same time fading in more red from 67mm up.

Disclaimer: I haven't tested this, except to run it against one Gcode file I had lying around. It seems like it generates the right code. Use at your own risk.

#!/usr/bin/perl -i
use strict;
use warnings;

# Declare some local variables
my $maxZ = 100;          # The height of the object being printed

# Ramp colour A from 100% down to 0 over the height of the object
# Ramp colour B from 0 to 75% over the height of the object
# When A+B < 1, colour C is used to "top up" to 1. This means that colour C starts appearing at 67mm height, and ramps up to 25% at the maxZ
my $startA = 1;             # Proportion of colour A at the first layer
my $endA = 0;              # Proportion of colour A at maxZ
my $startB = 0; 
my $endB = 0.75;

# Go through the input file, line by line
while ( <> ) {
	# Print the existing line to the output
	print;
	if (my $match = m/Z(\d+\.?\d*)/) {
		# If the line contains a Z followed by a number, then there's a height change, so insert a colour change M567 command
               # The number is extracted and stored in the Perl variable $1

               # Calculate the colour values
               # This is a simple linear interpolation so that colours A & B ramp from their start to end values from 0 to the height of the object ($maxZ)
		my $colourA = $startA + ( $endA-$startA )*$1/$maxZ;
		my $colourB = $startB + ( $endB-$startB )*$1/$maxZ;
		my $totalColour = $colourA + $colourB ;
                my $colourC = 0;
               # Ensure that the total of the colours is always equal to 1
		if ($totalColour >= 1) {
			$colourA /= $totalColour;
			$colourB /= $totalColour;
		} else {
		        $colourC = 1 - ( $colourA + $colourB );
                }

               # Output the M567 line. Each colour proportion is output with 2 decimal places
		printf "M567 P4 E%4.2f:%4.2f:%4.2f\n", $colourA, $colourB, $colourC;
	}
}

Edited 3 time(s). Last edit at 03/30/2016 05:05PM by frankvdh.

Thanks guys.

I should have explained that I am a sexagenarian carpenter who is just playing around with this stuff as a hobbyist. I can manage writing simple programs for arduinos and I design simple things using OpenScad but this is all self taught. It looks like I have something else to teach myself.

Since starting this post I have managed to write a macro using notepad++ which at least finds the next occurrence of "Z" and inserts a new line with the command M567 T3 E0.00:0.00:0.00. I did this on a 90mm high object with 0.3mm layer change. That's as far as my current knowledge would take me so then, feeling in a particularly masochistic mood, I went through the file and manually edited every one of the 300 instances to give me 1% changes, firstly from colour A to colour B, then colour B to colour C and then colour C to colour A. It'll be interesting to see how it prints but I value my sanity too much to go through that process again.

Thanks for pointing me at various options. It'll be a question of which one has the shallowest learning curve.

Edited 1 time(s). Last edit at 03/30/2016 05:04AM by deckingman.

I (a cinqagenarian programmer, also just playing around here as a hobbyist) am happy to help out with programming stuff. I have a Diamond hotend which I haven't yet hooked up to a printer, so this is very relevant and useful for me. I had no idea the M567 command existed, so suddenly there's a world of possibilities!

Perl is convenient, but a bit difficult to read... if there's some other programming language that you're comfortable with, I'm happy to switch to that. OTOH, Perl isn't impossible, and there's lots of online tutorials and help. To help, I've added comments to the code above.

Regular Expressions like /Z(\d+\.?\d*)/ look intimidating, but once you look into them, they're not that difficult, and very useful. But we could do this entirely without RegExs if you prefer.

I'm happy to continue here, or to go to email or PM.

This is all good stuff.

I looked around and settled on Python, for no better reason than there seemed to be more available in terms of tutorials and code snippets. I appreciate that it's not primarily designed for editing text files but I thought that learning to write code in Python would be a good skill to have for future hobby projects. Anyway, I've managed to write a small script that does what I set out to do along the lines of my opening post and your kind suggestions. It has 3 variables FilA, FilB and FilC. FilA starts at 1.00 and the other two at 0.00. There is another variable called "Iterations" which starts at 1. The script (program) reads a file line by line and writes each line to a new file. As it reads each line it looks for "G1 Z" and if it finds it, it adds a new line ("M567 P3 E"+'{0:.2f}'.format(FilA)+":"+'{0:.2f}'.format(Fil+":"+'{0:.2f}'.format(FilC)+"\n") to the output file. The formatting ensures that the output is always 2 decimal places because I discovered that taking 0.1 off of a floating point number repeatedly results in a very long number of decimal places (I remember reading something about that but can't remember the reason why). It might not matter but the resultant gcode file looks horrible. Anyway, it then reduces FilA by 0.01 and increases FilB by 0.01. If FilA<0 then it sets it to 0 and sets FilB to 1.00. Then it sets the Iteration to 2. If the iteration is 2 then it carries on through reducing FilB and increasing FilC until FilB reaches 0 at which point it changes Iteration to 3. Then it reduces FilC and increases FilA. When FilC reaches 0 it changes the iteration back to 1. So basically it "fades" the filaments from 100% A and 0% B through to 0% A and 100% B, then 100% B and 0% C to 0% B and 100% C, then 100%C and 0%A to 0%C and 100%A and in theory after 300 layers, it will start again with 100%A and 0%B.

The input file is opened as read only and the output fie is opened as write only. So, I can play around with a "live" gcode file without worrying about corrupting it. I figured that if I keep the same name for the output file then all I need is rename it when I'm happy with the changes.

Maybe it would be clearer if I posted the code but I don't know how to do that. Can you tell me how?

I'm also happy to continue here or go to email or PM. I would value your coding expertise and, having physically grafted a Diamond hot end onto my printer there are some tips and tricks that I've learned which I'm happy to share.

Edit, I see that "FilB" comes out as a smiley but don't know how to put that right.

Edited 2 time(s). Last edit at 03/31/2016 04:51AM by deckingman.

Here's my previous script, recoded into Python v3.5.1.

Edit: I'd misunderstood the Perl example in the Slic3r documentation, thinking it was reading stdin. But Perl's -i command-line parameter is kinda funky. To make this work in Python, I've made a few changes from the first version (hopefully no-one is trying to use that):
NB:

• I've (now) tested this with Slic3r 1.2.9, and it works
• Input filename is passed via argv[1] on the command line by Slic3r
• A temporary file is created, with the same name as the input filename, but with ".postproc" appended. When everything is done, the original file is deleted, and the temporary file is renamed to have the input file's name.
• In my second edit, I've added a few lines of code to write some stuff to a log file. I was rather hoping that Slic3r would tell me the height of the object via the environment variables, but there's nothing useful there.
• Uses a regular expression to find Z height
• Does the same calculations as my previous Perl script, not what you described... feel free to use it as a template for what you want. But file handling is now different from my Perl script.
• To run this from the command line for testing I use
  

   python35 [script].py [data].gcode

  Replace [script] by the script filename, replace [data] by the name of your sample gcode file. If you're on Windows, the installer installs Python 3.5.1 with the executable named "python", not "python35".
  Be aware that this replaces the input file, so if you run it twice, you'll end up with 2 sets of M567 commands for every Z
• To post your code here, open the file in NotePad++, press Ctrl-A to select all of it, and then Ctrl-C to copy it (or use NP++'s "Edit" menu to "Select-All" and "Copy"). Then switch to your browser where you have the forum open. Begin creating/editing a message. Click on the 15th icon in the message editor window (looks like a white square), This will put code and /code in square brackets in the editing window. Right-click between them, and choose "Paste"
• The smiley thing is because the forum interprets B followed by ) as a smiley . Use the Preview button to see where it happens. Insert a space between the B and ) to prevent that.

#!/usr/bin/python

import sys
import re
import os

# Declare some variables
maxZ = 100          # The height of the object being printed

# Ramp colour A from 100% down to 0 over the height of the object
# Ramp colour B from 0 to 75% over the height of the object
# When A+B < 1, colour C is used to "top up" to 1. This means that colour C starts appearing at 67mm height, and ramps up to 25% at the maxZ
startA = 1             # Proportion of colour A at the first layer
endA = 0               # Proportion of colour A at maxZ
startB = 0 
endB = 0.75

# Write metadata to a log file
log = open(sys.argv[1]+".log","w")
log.write(str(len(sys.argv)) + ' arguments.\n')
for v in sys.argv :
    log.write(v + "\n")

for v in os.environ :
    log.write(v + ": " + os.getenv(v) + "\n")

infile = open(sys.argv[1])
outfile = open(sys.argv[1]+".postproc","w")

# Read the input file
for line in infile :
	# Print the existing line to the output, without adding another newline
    outfile.write(line)
    match = re.search(r"Z(\d+\.?\d*)", line)
    if match :
        # If the line contains a Z followed by a number, then there's a height change, so insert a colour change M567 command
        # The number extracted is stored in match.group(1)
        height = float(match.group(1))
		
        # Calculate the colour values
		
		# Calculate proportions of colours A & B based on their linear ramps
        # This is a simple linear interpolation so that colours A & B ramp from their start to end values from 0 to the height of the object ($maxZ)
        colourA = startA + ( endA-startA )*height/maxZ
        colourB = startB + ( endB-startB )*height/maxZ
        totalColour = colourA + colourB

        # Ensure that the total of the colours is always equal to 1
        colourC = 0
        if totalColour >= 1 :
            colourA /= totalColour
            colourB /= totalColour
        else :
            colourC = 1 - ( colourA + colourB )

        # Output the M567 line. Each colour proportion is output with 2 decimal places
        outfile.write("M567 P4 E{0:.2f}:{1:.2f}:{2:.2f}\n".format(colourA, colourB, colourC))
# Close the files so that the OS can do things with them
outfile.close()
infile.close()
# Delete the original input file
os.remove(sys.argv[1])
# Rename the output file to be what the input file was called
os.rename(sys.argv[1]+".postproc", sys.argv[1])

Edited 3 time(s). Last edit at 03/31/2016 08:53PM by frankvdh.

Here is what I've managed to cobble together

startfile = open("D:\RepRap\G Code Files Diamond hot end\Python test file.gcode",'r')
endfile = open("D:\RepRap\G Code Files Diamond hot end\Python output.txt", 'w')

FilA=1.00
FilB=0.00
FilC=0.00
Iteration =1


LayerChange="G1 Z" 


numLines=sum(1 for line in (startfile))
               
startfile.seek(0) # make sure we start at the beginning of the file


# use the following to limit part of file?? Check the file length (number of lines, and set x to this at start

for x in range (0,numLines):
    
#for line in startfile:
    # if I use the above, it misses every other line but for x in range works ok.

    LineText=(startfile.readline())
    # uncomment the following line with caution - it really slows things down.
    # print (LineText) 
    endfile.write(LineText)
    if LayerChange in LineText:
        TextToWrite =("M567 P3 E"+'{0:.2f}'.format(FilA)+":"+'{0:.2f}'.format(Fil+":"+'{0:.2f}'.format(FilC)+"\n")
        endfile.write (TextToWrite)
        print (TextToWrite)
        if Iteration == 1: 
            FilA=FilA-0.01
            FilB=FilB+0.01
            FilC=0.00
            if FilA<0:
                FilA=0.00
                FilB=1.00
                Iteration =2
        if Iteration==2:
            FilA=0.00
            FilB=FilB-0.01
            FilC=FilC+0.01
            if FilB<0:
                FilB=0.00
                FilC=1.00
                Iteration=3
        if Iteration==3:
            FilC=FilC-0.01
            FilA=FilA+0.01
            FilB=0.00
            if FilC<0:
                FilC=0.00
                FilA=1.00
                Iteration =1
                
startfile.close()
endfile.close()

I think I prefer doing it this way. That is, slice as normal then generate a new file and leave the original untouched. This will allow me to do variations of the same object without having to re-slice. All I have to remember is to rename the output file to "something or other.gcode" otherwise it'll get overwritten each time.

One thing I don't understand is that if I use "for line in startfile:" as the start of the loop, the output file misses every other line. If I first check the number of lines in the file by doing numLines=sum(1 for line in(startfile)) then use "for x in range(0,numlines):" it works fine with no missed lines.

If you are interested I've just posted some pics of my first 300 colour object here [forums.reprap.org]

Regarding your preference for creating a new file: I understand that, and it makes sense. It would be worth looking at making this a standalone app with a GUI to specify colour bands, etc. If you use the date & time as part of the endfile filename, you won't accidentally overwrite an existing file. Or you could use this (untested by me) instead of "endfile =open()" to check whether the outfile already exists:

  import os.path
endname = "D:\RepRap\G Code Files Diamond hot end\Python output.txt"
if os.path.isfile(endname) :
    print("File " + endname + " already exists"
    quit()
endfile = open(endname, 'w')

Regarding the missing lines: I suspect that you had both these lines in your code:

for line in startfile:
    LineText=(startfile.readline())

The two lines should be replaced by one....

for LineText in startfile:

"for line in startfile" reads a line from the file into the variable line by doing a "line = startfile.readline()", plus checking to see if it has reached the end of file. There's also a 50/50 chance that the program would crash trying to read past the end of file.

Regarding your print!
Awesome!

It's kindof reminiscent of the days of 256 colour VGA displays. (Back then, I though they were awesome too!)

The poor mixing is a bit disappointing though. I wonder if tapping a thread into the end of the Diamond nozzle would improve that? (Not something I'm going to try though... a bit to expensive if it goes wrong).

And finally... how about extending this idea to change colour in the X & Y directions as well? This wouldn't work well for an object with long straight lines, but a circular layer is broken up into lots of short straight lines by the slicer, so should be quite amenable to this. If you want to try this, I'll be happy to put together some code to do it. I'm thinking of circles of contrasting colours, fading towards the centre, on the front, back, and sides of your design, but happy to go with whatever you like. I suspect that there might be a bit of filament wasted through this, but we are in the "Experimental" forum here! OTOH, you're paying for the filament, so I'm happy if you don't want to do this.

Edited 1 time(s). Last edit at 04/01/2016 04:01PM by frankvdh.

Thanks for the info and helping me out with the missing lines thing.

Ref the "poor" mixing, yes in a way it's disappointing but on the other hand it does lead to some other interesting colour variations. I quite like the fact that the object looks different depending on the viewing angle. When I get chance I'll take some pics of the 3 sided pyramid with each face a different colour.

Ref extending the idea in the x and y directions, yes of course. From my point of view, this whole exercise so far is just scratching the surface of what might be possible. One thing that might screw it up though is that it takes a few mm of filament to purge the mixing chamber of what is already in there and then it's a gradual transition, not a sudden colour change. In practice, setting the perimeters to 2 or 3, it always starts with the inner perimeters before doing the outer perimeter so the purging happens on the inside and is completed before the outside perimeter starts. This may not be the case on small objects of course.

Let me know when you are about to hook up your diamond hot end as there are a couple of tricks I've learned along the way.

My day job of cutting and screwing bits of wood together is starting to get busy as it normally does at this time of year, so I won't have as much time to play for a few months (still got nearly 3 years before I'll get the state pension - not that that will be enough to retire on ).

I am tying to modify my g-code print files to change tools (different extruders). I am using the RoStock V2 delta with the Duet and Duex4 with the Diamond print head. i am able to easily print with any of the three extruders and can change tools (extruders) with the pause and resume command, but have had trouble putting the code into the print file gcode. As noted below I can sent the M567 command with out any effect.

Hoping someone can help. The Python script to define mixing is awesome by the way!!!!

thanks,

Dan
..................

Of note: The successes!!!!!!

• If I pause and then change tools by clicking on the web-interface "heater 1" and then selecting another tool, this now works to change the extruder and color. i think this now works since I reset my standby temp on my to my active temp (i.e. G10 P0 S228 R228). I think this is because each tool shares the same heater, thus tool 1 will never reach standby temp and thus, allow tool 2 to start....... Agree???? Correct!!!
• If I change T0 to T1 in the first lines of the sliced g-code, this starts the print with the appropriate tool color and works beautifully.

My remaining issues:

• I can send the M567 P1 or M567 P2 command during a print and it will acknowledge the command (little blue box) and show the mixing ratios, however will not change the tool. This does absolutely nothing to the print. I have attached the zip file of the calibration print if you want to review.
• If I sent a T# (1, 2, 3. 4. 5) in the command line, the printer simply stops and I have to reset via an "Emergency Stop" to restart.

Edited 1 time(s). Last edit at 04/14/2016 10:43PM by 3D-Print.

See my reply to your other post but I suspect that you haven't enabled mixing in your config.g

Here's a program I came up with for processing gcode:
Just place the file in the same folder as the executable.
It basically just creates a gradient from one height to the next. It also gets rid of having to use firmware retraction.

[drive.google.com]