November 2022

The Making of TS-MPPT Remote Monitor Page in Python with Graphs, Part 2: Generating HTML and Canvas Gauges

November 30, 2022 Kirk Comments 0 Comment

Today’s programming/debug time will be spent on adding in the code that generates the HTML page and adds in the Canvas gauges.

So first thing to do is to copy the Canvas Gauges Java Script into the program working directory since this will be the base directory for the HTML pages. I have attached the 7Zip file here or you can get it from here: https://canvas-gauges.com/download/ if you prefer.

gauge.min_.js Download

The next step is to put the code that will generate the HTML page that is mostly borrowed code from the QuickCPM.py program that has been stripped down to just one gauge to start with:

def getGaugePage(Voltage,Current,Amp_hrs,Gauge_size,Graph_width,Graph_height):
    hpg = "<html><head><meta http-equiv=\"refresh\" content=\"10\"><meta http-equiv=\"Cache-Control\" content=\"no-store\" />"#"refresh\" content=\"60\"
    hpg += "<meta charset=\"UTF-8\"/><title>TS-MPPT Monitor</title>"
    hpg += "<style>._PR {font-size:15px;box-shadow: 4px 4px 4px black;color:#333;border-top-left-radius: 8px;border-top-right-radius: 8px;border-bottom-right-radius: 8px;border-bottom-left-radius: 8px;border-width:1px;border-style: solid;border-color:silver;text-align: left;margin:4px 4px 4px 4px;padding:4px 4px 4px 4px;}</style>"
    hpg += "<script src=\"gauge.min.js\"></script>"
    #hpg += "<link rel=\"icon\" href=\"icon.png\">"
    hpg += "<div>"
    hpg += gaugeVoltageScrpt("Volts",Voltage,"Volts","Volts",Gauge_size) 
    hpg += "</a>"
    hpg += "</div>"
    hpg += "</div>"
    hpg += "<div>"
       #future spot for buttons
    hpg += "</div>"
    hpg += "</div>"
    hpg += "</div>"
    hpg += "<div></body></html>"
    return(hpg)

And then the next step will be to add in the first gauge. Editing the gauges are almost self explanatory. For help on the settings just Google ‘Canvas Gauge settings’ there are plenty of great example out there. As for the colors I use https://rgbacolorpicker.com/ to quickly pick the colors that I would like to have.

## Voltage gauge 21V-32V for 24V system

def gaugeVoltageScrpt(id,value,title,unit,size):
    ghpg = "\n\n<canvas id=\"gauge-ps{0}\"></canvas>\n".format(id)
    ghpg += "<script>\n"
    ghpg += "var gaugePStemp = new RadialGauge({\n"
    ghpg += "renderTo: 'gauge-ps{0}',\n".format(id)
    ghpg += "width:{0},\n".format(size)
    ghpg += "height:{0},\n".format(size)
    ghpg += "units: '{0}',\n".format(unit);
    ghpg += "minValue: 21,\n"
    ghpg += "maxValue: 32,\n"
    ghpg += "majorTicks: [\n"
    ghpg += "'21',\n"
    ghpg += "'22',\n"
    ghpg += "'23',\n"
    ghpg += "'24',\n"
    ghpg += "'25',\n"
    ghpg += "'26',\n"
    ghpg += "'27',\n"
    ghpg += "'28',\n"
    ghpg += "'29',\n"
    ghpg += "'30',\n"
    ghpg += "'31',\n"
    ghpg += "'32',\n"
    ghpg += "],\n"
    ghpg += "minorTicks: 5,\n"
    ghpg += "ticksAngle: 280,\n"
    ghpg += "startAngle: 40,\n"
    ghpg += "strokeTicks: true,\n"
    ghpg += "highlights  : [\n"
    ghpg += "{ from : 21,  to : 22.8, color : 'rgba(230, 47, 52, 0.8)' },\n"
    ghpg += "{ from : 22.8,  to : 28.8, color : 'rgba(58, 226, 147, 0.8)' },\n"
    ghpg += "{ from : 28.8,  to : 31.8, color : 'rgba(230, 136, 53, 0.8)' },\n"
    ghpg += "{ from : 31.8,  to : 32, color : 'rgba(230, 47, 52, 0.8)' },\n"
    ghpg += "],\n"
    ghpg += "valueInt: 1,\n"
    ghpg += "valueDec: 2,\n"
    ghpg += "colorPlate: \"#fff\",\n"
    ghpg += "colorMajorTicks: \"#686868\",\n"
    ghpg += "colorMinorTicks: \"#686868\",\n"
    ghpg += "colorTitle: \"#000\",\n"
    ghpg += "colorUnits: \"#000\",\n"
    ghpg += "colorNumbers: \"#686868\",\n"
    ghpg += "valueBox: true,\n"
    ghpg += "colorValueText: \"#000\",\n"
    ghpg += "colorValueBoxRect: \"#fff\",\n"
    ghpg += "colorValueBoxRectEnd: \"#fff\",\n"
    ghpg += "colorValueBoxBackground: \"#ddd\",\n"
    ghpg += "colorValueBoxShadow: true,\n"
    ghpg += "colorValueTextShadow: true,\n"
    ghpg += "colorNeedleShadowUp: true,\n"
    ghpg += "colorNeedleShadowDown: true,\n"
    ghpg += "colorNeedle: \"rgba(200, 50, 50, .75)\",\n"
    ghpg += "colorNeedleEnd: \"rgba(200, 50, 50, .75)\",\n"
    ghpg += "colorNeedleCircleOuter: \"rgba(200, 200, 200, 1)\",\n"
    ghpg += "colorNeedleCircleOuterEnd: \"rgba(200, 200, 200, 1)\",\n"
    ghpg += "borderShadowWidth: 4,\n"
    ghpg += "borders: true,\n"
    ghpg += "borderInnerWidth: 0,\n"
    ghpg += "borderMiddleWidth: 0,\n"
    ghpg += "borderOuterWidth: 5,\n"
    ghpg += "colorBorderOuter: \"silver\",\n"
    ghpg += "colorBorderOuterEnd: \"#cdcdcd\",\n"
    ghpg += "needleType: \"arrow\",\n"
    ghpg += "needleWidth: 2,\n"
    ghpg += "needleCircleSize: 7,\n"
    ghpg += "needleCircleOuter: true,\n"
    ghpg += "needleCircleInner: false,\n"
    ghpg += "animationDuration: 2000,\n"
    ghpg += "animationRule: \"dequint\",\n"
    ghpg += "fontNumbers: \"Verdana\",\n"
    ghpg += "fontTitle: \"Verdana\",\n"
    ghpg += "fontUnits: \"Verdana\",\n"
    ghpg += "fontValue: \"Led\",\n"
    ghpg += "fontValueStyle: 'italic',\n"
    ghpg += "fontNumbersSize: 20,\n"
    ghpg += "fontNumbersStyle: 'italic',\n"
    ghpg += "fontNumbersWeight: 'bold',\n"
    ghpg += "fontTitleSize: 24,\n"
    ghpg += "fontUnitsSize: 22,\n"
    ghpg += "fontValueSize: 35,\n"
    ghpg += "animatedValue: true\n,"
    ghpg += "});\n"
    ghpg += "gaugePStemp.draw();\n"
    ghpg += "gaugePStemp.value = \"{0}\";\n".format(value)
    ghpg += "gaugePStemp.title = \"{0}\";\n".format(title)
    ghpg += "</script>\n"
    return ghpg
    #The Mad Scientist Hut.

Then the call to the getGaugePage routine gets added into the program for the testing of the py code with a real call to read data from the TS-MPPT-60 then display it on the gauge page

        writeToFile("Solar.html",getGaugePage(Battery_Voltage,Charge_Amps,Amp_hrs,Gauge_size,Graph_width,Graph_height),0,1)

This is what the generated web page looks like at this point

Canvas Gauge for solar system monitor generated by Python

Okay the Voltage gauge looks good, time to add in the Amperage gauge. In the def getGaugePage add a call to the new def gaugeAmperageScrpt just below the Voltage Gauge call

    hpg += "<div>"
    hpg += gaugeVoltageScrpt("Volts",Voltage,"Volts","Volts",Gauge_size) 
    hpg += gaugeAmperageScrpt("Amps",Current,"Amps","Amps",Gauge_size)

And add in the Amperage gauge code

## Amperage gauge 0-65A the TS-MPPT-60 can source a hair over 60 Amps at peak

def gaugeAmperageScrpt(id,value,title,unit,size):
    ghpg = "\n\n<canvas id=\"gauge-ps{0}\"></canvas>\n".format(id)
    ghpg += "<script>\n"
    ghpg += "var gaugePStemp = new RadialGauge({\n"
    ghpg += "renderTo: 'gauge-ps{0}',\n".format(id)
    ghpg += "width:{0},\n".format(size)
    ghpg += "height:{0},\n".format(size)
    ghpg += "units: '{0}',\n".format(unit);
    ghpg += "minValue: 0,\n"
    ghpg += "maxValue: 65,\n"
    ghpg += "majorTicks: [\n"
    ghpg += "'0',\n"
    ghpg += "'5',\n"
    ghpg += "'10',\n"
    ghpg += "'15',\n"
    ghpg += "'20',\n"
    ghpg += "'25',\n"
    ghpg += "'30',\n"
    ghpg += "'35',\n"
    ghpg += "'40',\n"
    ghpg += "'45',\n"
    ghpg += "'50',\n"
    ghpg += "'55',\n"
    ghpg += "'60',\n"
    ghpg += "'65',\n"
    ghpg += "],\n"
    ghpg += "minorTicks: 5,\n"
    ghpg += "ticksAngle: 280,\n"
    ghpg += "startAngle: 40,\n"
    ghpg += "strokeTicks: true,\n"
    ghpg += "highlights  : [\n"
    #ghpg += "{ from : 21,  to : 22.8, color : 'rgba(230, 47, 52, 0.8)' },\n" #going to add in a color mark for peak amperage over last hour later
    ghpg += "{ from : 0,  to : 65, color : 'rgba(173, 196, 250, 0.8)' },\n"
    ghpg += "],\n"
    ghpg += "valueInt: 1,\n"
    ghpg += "valueDec: 2,\n"
    ghpg += "colorPlate: \"#fff\",\n"
    ghpg += "colorMajorTicks: \"#686868\",\n"
    ghpg += "colorMinorTicks: \"#686868\",\n"
    ghpg += "colorTitle: \"#000\",\n"
    ghpg += "colorUnits: \"#000\",\n"
    ghpg += "colorNumbers: \"#686868\",\n"
    ghpg += "valueBox: true,\n"
    ghpg += "colorValueText: \"#000\",\n"
    ghpg += "colorValueBoxRect: \"#fff\",\n"
    ghpg += "colorValueBoxRectEnd: \"#fff\",\n"
    ghpg += "colorValueBoxBackground: \"#ddd\",\n"
    ghpg += "colorValueBoxShadow: true,\n"
    ghpg += "colorValueTextShadow: true,\n"
    ghpg += "colorNeedleShadowUp: true,\n"
    ghpg += "colorNeedleShadowDown: true,\n"
    ghpg += "colorNeedle: \"rgba(200, 50, 50, .75)\",\n"
    ghpg += "colorNeedleEnd: \"rgba(200, 50, 50, .75)\",\n"
    ghpg += "colorNeedleCircleOuter: \"rgba(200, 200, 200, 1)\",\n"
    ghpg += "colorNeedleCircleOuterEnd: \"rgba(200, 200, 200, 1)\",\n"
    ghpg += "borderShadowWidth: 4,\n"
    ghpg += "borders: true,\n"
    ghpg += "borderInnerWidth: 0,\n"
    ghpg += "borderMiddleWidth: 0,\n"
    ghpg += "borderOuterWidth: 5,\n"
    ghpg += "colorBorderOuter: \"silver\",\n"
    ghpg += "colorBorderOuterEnd: \"#cdcdcd\",\n"
    ghpg += "needleType: \"arrow\",\n"
    ghpg += "needleWidth: 2,\n"
    ghpg += "needleCircleSize: 7,\n"
    ghpg += "needleCircleOuter: true,\n"
    ghpg += "needleCircleInner: false,\n"
    ghpg += "animationDuration: 2000,\n"
    ghpg += "animationRule: \"dequint\",\n"
    ghpg += "fontNumbers: \"Verdana\",\n"
    ghpg += "fontTitle: \"Verdana\",\n"
    ghpg += "fontUnits: \"Verdana\",\n"
    ghpg += "fontValue: \"Led\",\n"
    ghpg += "fontValueStyle: 'italic',\n"
    ghpg += "fontNumbersSize: 20,\n"
    ghpg += "fontNumbersStyle: 'italic',\n"
    ghpg += "fontNumbersWeight: 'bold',\n"
    ghpg += "fontTitleSize: 24,\n"
    ghpg += "fontUnitsSize: 22,\n"
    ghpg += "fontValueSize: 35,\n"
    ghpg += "animatedValue: true\n,"
    ghpg += "});\n"
    ghpg += "gaugePStemp.draw();\n"
    ghpg += "gaugePStemp.value = \"{0}\";\n".format(value)
    ghpg += "gaugePStemp.title = \"{0}\";\n".format(title)
    ghpg += "</script>\n"
    return ghpg

A quick test run of the code and this is what the page now:

I guess this is a good stopping point for today. Tomorrow I think it will be time to start adding in the graphs.

The Making of TS-MPPT Remote Monitor Page in Python with Graphs, Part 1: Creating the Data Log CSV file

November 29, 2022 Kirk Comments 0 Comment

I have had a Tristar Maximum Power Point Tracker ( TS-MPPT-60 ) made by Morningstar in my solar system for several years. There is a webpage server program built into the charge controller, that displays information on the controller. The charge controller pages can only be accessed while connected to the home network, as it lives inside of my internet firewall and will remain this way for security reasons. While the built in page displays key parameters, it only shows a snap shot of the current time and has a data log function that only shows a single point of key data totals for each day of operation. I really would love to see real time data and long term data charted out in graphs presented on a live webpage that can be accessed from anywhere. This project will implement a web page display somewhat in a very similar style to what was done in the GMC-320 webpage from my previous blog entry.

Note: The current project live monitor webpage can be viewed here : https://madscientisthut.com/TSMPPT_monitor/Solar.html

Here is what the Morningstar TS-MPPT-60 built in liveview page looks like:

I have started to play with Python programming a lot since starting the GMC-320 webpage monitor project back in October. My interest was really peaked when I saw that Python had such great power such as stripping data from HTML. Wondering if that data could be stripped from the built in web sever on the charge controller I started down this path only to find the data in the status page was provided by a built in java script. After futilely messing around with it for a couple of hours I searched online and found a blog written by Takashi Ando here: https://tech.blog.uribou.me/python-driver-for-tsmppt60/ where he had done all of the leg work for getting data back from the Charge controller. No need to reinvent the wheel, thank you so much Takashi for making this great starting point for my next new project!

This Mad Scientist Hut project will be documented in this blog as progress is completed.

A great start: After downloading the driver from Takashi’s Git hub and with these few little tweaks: commented out the SolarArrayStatus, TemperaturesStatus, and OperatingConditions from the Class:SystemStatus the program is already getting data back that is needed for the project. The main data of interest is Battery Voltage, Charging Amps and Amp Hours. A side note, it is snowing today so the solar panels are covered in tons of snow, no charge current today ☹️

My python script to read back from Takashi’s driver:

print("Read…")
tsmppt_dict = SystemStatus("tsmppt12400032").get()
print (tsmppt_dict)

Terminal output:

Read…
{'Battery Voltage': {'group': 'Battery', 'value': 24.69, 'unit': 'V'}, 'Target Voltage': {'group': 'Battery', 'value': 31.64, 'unit': 'V'}, 'Charge Current': {'group': 'Battery', 'value': 0.04, 'unit': 'A'}, 'Amp Hours': {'group': 'Counter', 'value': 224726.5, 'unit': 'Ah'}, 'Kilowatt Hours': {'group': 'Counter', 'value': 6219, 'unit': 'kWh'}}
Done

Hmmm… I have never encountered dictionary variables before in coding, this is different. Well let me see what can be done with this. Okay not so bad, after reading up on nested dictionary variables this is what works:

battery_value = float(tsmppt_dict['Battery Voltage']['value'])
amps_value = float(tsmppt_dict['Charge Current']['value'])
amp_hours_value = float(tsmppt_dict['Amp Hours']['value'])

Now to store the data in a CSV format. This is code from the QuickCPM.py that is being copied over for the CSV file and also later for the HTML file writes.

from pathlib import Path #<--put this with the other imports

def writeToFile(fileName,data, isbinary, ishtml):
        if len(data)>1:
            fn= fileName # QtGui.QFileDialog.getSaveFileName(self, "Save file", "", "*.*")with open("test.txt", "a") as myfile:
            if isbinary:
                #print "Writing binary"
                with open(fn, "wb") as file:  
                    file.write(data)  
                #print "{:d} bytes saved to {:s}".format(len(data), fn)
            else:
                if ishtml:
                    with open(fn, "w") as file:  
                        file.write(data)
                        #print "{:d} chars written to {:s}".format(len(data),fn)
                else:
                    with open(fn, "a") as file:  
                        file.write(data)
                        #print "{:d} chars written to {:s}".format(len(data),fn)
                    
        else:
            print ("No data to save")

Next step put the tsmppt data read into a call (allows for multiple charge controllers by address later)

def readTSMPPTdata(address):
    try: 
        tsmppt_dict = SystemStatus(address).get()
        battery_value = float(tsmppt_dict['Battery Voltage']['value'])
        amps_value = float(tsmppt_dict['Charge Current']['value'])
        amp_hours_value = float(tsmppt_dict['Amp Hours']['value'])
        cl = "{:%y-%m-%d %H:%M}".format(datetime.datetime.now())
        Data = "{0},{1},{2},{3}\n".format(battery_value,amps_value,amp_hours_value,cl)
    except:        
        print ("readTSMPPTdata Error")
    return (Data)

then start a loop to test the CSV file

try:
    #Write log file headers for columns if CSV is not written yet
    file_to_check = Path(Data_CSV_FileName)
    if not file_to_check.is_file():
        cl = "{:%y-%m-%d %H:%M}".format(datetime.datetime.now())
        Data_for_CSV = "Battery Voltage,Charge Current, Total Amp Hours,Date Time,,Data_file_started_at: {0}\n".format(cl)
        writeToFile(Data_CSV_FileName,Data_for_CSV,0,0)

    while True:
        Data_for_CSV = readTSMPPTdata("tsmppt12400032")
        writeToFile(Data_CSV_FileName,Data_for_CSV,0,0)
  
except KeyboardInterrupt:
    pass

Contents of the CSV:

Okay time to take a break. I will continue this tomorrow. The next step is to add the code that will generate the webpage and add in the canvas gauges.

GMC-320 Radiation Web Page Monitor

November 19, 2022 Kirk Comments 0 Comment

I purchased a GQ GMC-320 Plus Geiger Muller Counter Data Logger from Amazon a few years ago.

The main reason I purchased the GQ Geiger counter was that it has a serial interface and what looked like interesting software from GQ especially graphs of data captured. I played with the GQ software but was not really impressed since you can only get an hour worth of CPM readings without paying for upgraded software from GQ so in the drawer the GMC-320 went….

I got bored last month and found the GMC-320 sitting in the drawer, decided to take it out and play with it a little. A couple of years have gone by so the software must be better by now. I downloaded the current interface from the GQ website and installed it on my laptop… nope same old interface, how very disappointing. Someone must have better software to interface the unit with, while searching I stumbled across this code written for a Raspberry Pi on the 247coding.com website. It had some very basic features that I liked such as it read data from the serial interface, it created a gauge web page, FTP upload and it stored data in a CSV file. Wow exactly almost everything I needed to get started.

This is what the original QuickCPM code webpage looked like:

QuickCPM.py Python CPM monitor for GMC-280 GMC-300 GMC-320 plus

Things I did not like about the original code:

It had a fixed com port on the Raspberry PI
only displayed CPM and temperature from the counter
no graphing
no error checking for bad data received
original code was from Python 2.x

Do not get me wrong, this was a great piece of code to start from, without this code I think the GMC-320 would be back in the drawer. Okay, so I have many challenges to overcome that I can tackle my boredom with.

I had never worked with Python before. I setup the Python 2.7.3 IDLE interface on my Win 10 laptop and started to play with the code. Python had some weird stuff to get used to but I have coded in several other languages so not surprisingly it was so simple pick up Python code and run with it. After playing around with the 247coding.com code for a few days and getting the code to run on Windows I decide it was time to make some changes.

Changes to the code include:

Added graph for Temperature
Added uSv information with CPM to uSv/hr calculation based on the M4011 Geiger Muller Tube at a 153.8 ratio.
Added graph for uSv per hour for each minute of captured data
Added uSv dose rate for each hour of captured data
Auto Com Port identification
If Com Port is disconnected, the loop will continue and reconnect even if the device is plugged into a different port
Coded to work in multiple operating systems for Com Ports ( currently only verified on Win 10 )
If serial data is in error or corrupted data received, reject data and try again
FTP setup to transfer gauge pages on each minute, new uSv per hour and temperature graph every ten minutes, and uSv hour dose rate every hour.
Modified code to work in Python 3.x
Added program start time, last update, peak CPM time to display
Added port connection to display
Added separate gauge for peak CPM
Changed gauges to have a better appearance, and changed range
Changed CSV files to add header to beginning row and to log all pertinent data
Setup two pages sizes, one for cell phone and one for desktop displays.
And many other little changes… from The Mad Scientist Hut

The code should work on GMC-280, GMC-300, and of course the GMC-320 which it was tested for. As for the 280 and 300 the baud rate will have to be changed in the code variables

So here is the what the new code web page looks like now:

GQ GMC-320 PLUS Web Page Monitor GMC-280 GMC-300 Python uSv/hr uSv total dose rate CPM Temperature

Note: On live page click the temperature gauge for temperature graph.

I am still playing with the code a little bit but am pretty much complete with what I wanted out of it.

I may add a separate page to this monitor with interactive data charts. The only issue I have with this is the interactive charts are fairly large chunks of data to transfer via FTP then again also to receive to a web page. So if I do this, I may set up the program to only transfer the interactive charts once every 24 hours. I really think it would be nice to have the interactive data charts for long term data but also think they may not need to be accessed very much.

Click GMC Radiation Monitor to see real time data from the working code. The unit should be up and running but may be down while I am working on the laptop. I am working on transferring this to an Acer Aspire One with a very low power atom processor running Linux, seeing how Raspberry Pi is still a fairly costly out of stock item for the time being…

Update: 11/23/22 — I have completed transferring this Python code from my Win 10 laptop to the Acer Aspire Netbook Nav450 with an Intel Atom processor. I have set the Acer netbook up to run Puppy Linux Fossapup64 from a USB stick. I was happy I only needed a couple of quick little fixes in the code to get this running on Linux. The main time spent was getting Puppy setup and installing all of the Python libraries, mainly because Puppy is a very lite version of Linux and it needs extra care and feeding to get it going nicely. A note for installing the Kaleido library into Python on Puppy on a machine with 1Gb of memory is to use export TMPDIR=’/var/tmp’ before running the pip3 install -U Kaleido command.

If you are interested in the code as it currently stands, ping me on my YouTube channel since it is the only place where I check messages anymore.

I wish to thank the original coder/s at 247coding.com for the starter code for this project!

This post contains a link to the Amazon web page GMC-320: As an Amazon Associate I earn from qualifying purchases.

For reference the GQ-RFC1201 GQ GMC Geiger Counter Communication Protocol manual is here.