Program to record data from a cheap OBD2 scanner

[RScott]

Last year, I spent about $30 for an OBD2 scanner, not sure of what (if anything) I would be able to do with it.

Needless to say, I have been quite impressed! With it, I was able to access the primary high speed GMLAN bus using DWCAN at 500kbps (but not the high speed chassis expansion bus or low speed GMLAN/SWCAN 33.6kbps buses). There is an incredible amount of data (speed, battery SOC, latitude/longitude, driver/passenger seatbelts fastened, shifter position (PRNDL), accelerator/brake position, etc.). To get an idea of what can be done if you start recording data, you can go to http://www.evtools.info/display3.htm.

The problem is that there doesn't seem to be software out there to let you easily record this data. So I wrote a program to do so (the main problem is that you have to set the speed to 500kbps, or else a lot of data would get lost). This program records the data into an easy-to-read (for a computer program) text file, that can later be analyzed. So even if you don't do anything with the data now, except record it, you can later use it for analysis. It is "dumb", in that it simply saves all data and does not interpret it (so everything will be recorded, regardless of whether or not it is known what the data does).

The program runs on Windows, so it requires a laptop/netbook running Windows. It works with the ELM327 v1.4 chipset, and requires a USB interface (not Bluetooth, which is not capable of the high speeds the Volt uses). The $30 ElmScan 5 at http://www.amazon.com/ElmScan-Compac.../dp/B002PYBZJO should do the trick. The program is just an executable file; hopefully, I'll be able to release the source as well at some point in the near future. You can download the executable online at http://www.evtools.info/exe/volt.exe.

It isn't very user friendly (but has worked flawlessly for me for many months now). It is designed to be run from a command prompt. It expects only one COM port. It then creates a unique file (based on the date and a 3-digit extension, e.g. 20120529001.txt for the first file for May 29, 2012), and then saves whatever it reads until the program is ended (e.g. with CTRL-C). Sleep mode may cause it to hang; if that happens, a reboot may be necessary to run the program again.

You should be able to start the program as you normally would, typing CTRL-C to end it. I normally just keep it running unless I expect to be away from the car long enough that the netbook would go to sleep.

Further posts will have more details, but this is enough to get started. Feel free to ask any questions.

[RScott]

Here are some further details.

The program starts by looking for a COM port (automatically connecting to the first one it finds, or waiting if none are found). It then tries communicating with the ELM327 at various bps rates (in case a different tool was used and left it at an unexpected speed). Once it finds the correct speed, it sends a warm start command, echo off command, linefeeds off command, and a spaces off command. It then switches to 500kbps, sends the command to show headers, then sends the "monitor all" command.

The data is saved into a text file where comment lines start with a '#'; otherwise, it records the time the data was received (HH:MM:SS.mmm), a colon and a space, the 3-digit CAN ID, and the CAN data (e.g. "09:23:13.384: 206 E31700"). It receives data in batches, so typically there will be several lines with the same timestamp.

The program is designed to be as accurate as possible in what it saves, doing as little interpretation as possible. One known bug is that it appears that the ELM327 chipset may not record all the bytes of data if the first byte of data is from 0x01 through 0x07 (thinking that the first byte of data indicates the number of bytes of data to follow), and I haven't found any way around that (fortunately, most of the useful data is not affected by this). The other bug I am aware of is that the ELM327 is limited to 500kbps, which sounds fine since the Volt uses a 500kbps bus, but since the ELM327 has a bit more overhead than the Volt, some data is occasionally lost (fortunately, that too is not normally an issue).

It also has the option to display some or most CAN IDs on the screen while the program is running by adding arguments with a "+" or "-" and a CAN ID (e.g. "Volt +206 +208" to show the values of the CAN IDs 206 and 208). I've used that to help track down what CAN IDs are used for certain features (e.g. opening the door).

I plan to start working on a slightly "cleaner" and easier-to-use version, and seeing if I can release the source code.

This program simply logs data; I am also working on another program that actually does something with the data (e.g. producing HTML files with a map, extrapolating data, playing back the data in real-time so you can see how the CAN data changes, etc.).

[mrosack]

That looks pretty awesome, just ordered an ElmScan!

It'd be sweet if we could log (and hopefully visualize) the data on an Android tablet. There's Torque: https://play.google.com/store/apps/d...l.torque&hl=en, but it requires a bluetooth connection. I don't think it's possible to talk to a USB serial port without a rooted Android device, but I'd buy a crappy cheepo Android tablet to root and keep in my car for this purpose...

[Guessin]

Do you have the pid and decodes that I can get for an iPhone app im workin on

[tboult]

Do you have the pid and decodes that I can get for an iPhone app im workin on

RScott is not using PID polling, he is doing passive scanning of the bus.
I've only found one PID for battery specific info


PID 01 5B
Type precent
Formula is
A*100/255

This gives raw battery SOC.


If you want battery as a usable percent (0=empty 100%=full) then try

PID 01 5B
min(100,max(0,(((A*100)/255-21)/65)))

Where the 21 is the lowest battery level you see and 86 is about the highest. (I;ve actually seen 20.5 and 87.6 but I think those were abnormal situations).

There are a lot of "normal" PIDs for engine, pedal and such I could provide if I go back and look for them (they were all just choices from standard GM PIDs for other chars).

[Automotive Motivation]

I just placed an order for the ElmScan too!

[wizkid057]

@RScott

I've also been using my own software to datalog the "monitor all" function from my ELM device.

Once thing I've noticed which you seem to have solved is with the GPS coordinates. The data I get does not seem to convert directly from milliarcseconds to archours for the longitude bytes. First, it should work out to a negative value for me, and the sign bit is not set. However, the high nibble of the high byte seems to have bits set (seems to always be 6 for me driving around). If I drop the high nibble and calculate out archours, its off by 1.08xxx archours or so, give or take, from my actual position. The latitude calculations work out perfectly (X / 3600000). Have you encountered this problem or is my sub-par ELM tool fumbling the bits somewhere?

Also, I'd be interested to know of any other data fields you've been able to make sense of. While my initial goal for this is to make a standalone device to interpret the data and display something like a kW meter and such (something that I think the Volt should have had to start with...), the more data the merrier!

On a related, but not directly, note, I've already been using my own software to poll OnStar Remotelink and database all of the data for some pretty nifty statistics.
Heres some info I can compile from my database: .
Yes, I've used about 1 megawatt hour of grid power so far!

I plan on eventually installing my Raspberry Pi in the Volt to continuously log the data and wirelessly transmit it to my server for databasing when in range of either manually known or some generic public wifi, which should be pretty dang cool.

Thanks,

-wk

[tboult]

SNIP

On a related, but not directly, note, I've already been using my own software to poll OnStar Remotelink and database all of the data for some pretty nifty statistics.
Heres some info I can compile from my database: .
Yes, I've used about 1 megawatt hour of grid power so far!

I plan on eventually installing my Raspberry Pi in the Volt to continuously log the data and wirelessly transmit it to my server for databasing when in range of either manually known or some generic public wifi, which should be pretty dang cool.

Thanks,

-wk

interesting.. what are you using to poll onstar.. I'd be interested in that type of data..

[wizkid057]

interesting.. what are you using to poll onstar.. I'd be interested in that type of data..

I basically did the same thing Voltstats did initially and reverse engineered the Remotelink app for Android.

The code I've done for it is definitely not pretty, but, it seems to work and has for several months now.

My database stores information in the following mySQL table by polling OnStar about every 15 minutes during the day, and every 30 minutes 12AM to 5AM.

create table voltstats (
   id INT AUTO_INCREMENT,
   date DATETIME,
   odometer DOUBLE,
   evodometer DOUBLE,
   batterylevel DOUBLE,
   fuellevel DOUBLE,
   oillife DOUBLE,
   lifempg DOUBLE,
   electricecon DOUBLE,
   evrange DOUBLE,
   gasrange DOUBLE,
   tirepsiLF DOUBLE,
   tirepsiRF DOUBLE,
   tirepsiLR DOUBLE,
   tirepsiRR DOUBLE,
   evplugstate VARCHAR(64),
   chargestate VARCHAR(64),
   evplugvoltage INT,
   lasttripmpg DOUBLE,
   lasttripevmiles DOUBLE,
   lasttriptotalmiles DOUBLE,
   ignitionstatus VARCHAR(64),
   direction FLOAT,
PRIMARY KEY(id));

From sequential rows of this data I can calculate out a lot of things, including average speed, fuel consumption, battery energy used, etc. My kWh tracking is based on changes in the reported battery level and a pretty straightforward formula to estimate that into kWh to the battery and from the wall, in the case of an increase. I've used the numbers from my Kill-a-Watt (for 120V charging) and my digital meter (on my 240V charger) to get an average of how much energy comes from the wall per battery %. (For me, turns out to be 100%=~12.5 kWh @ 240V and ~13.75kWh @ 120V, for those interested.)

Not to hijack the thread, but, heres a link to some of the graphs I generate from the data:


http://www.wizkid057.com/volt/

Perhaps when I get some more time I'll tidy and publish the code, although, I'm not sure how much OnStar would appreciate that...

-wk

[RScott]

Once thing I've noticed which you seem to have solved is with the GPS coordinates. The data I get does not seem to convert directly from milliarcseconds to archours for the longitude bytes. First, it should work out to a negative value for me, and the sign bit is not set. However, the high nibble of the high byte seems to have bits set (seems to always be 6 for me driving around). If I drop the high nibble and calculate out archours, its off by 1.08xxx archours or so, give or take, from my actual position. The latitude calculations work out perfectly (X / 3600000). Have you encountered this problem or is my sub-par ELM tool fumbling the bits somewhere?

Here's how I convert the 4-byte latitude or longitude to a float ('mas' is milliarcseconds as an unsigned int):

if ( mas >= 0x40000000 )
{
	mymas = 0 - ( 0x80000000 - mas );
}

f = ( float )mymas / 3600000;

As a sample to test with, if I get 0x912cd7a for the latitude and 0x70a26c9b for the longitude, I get 42.285332 x -71.608559 (and if you enter that into Google Maps, it should be a FedEx Office print and ship center).

Other than what is in the calculation above, I haven't encountered any oddities with the lat/lon bytes.

Also, I'd be interested to know of any other data fields you've been able to make sense of.

I have my initial results at http://www.evtools.info/ChevyVoltOBD2CAN.html . I've come up with some more, but haven't updated that page in quite some time. But most of the best stuff is already there.

It would be nice if OnStar would release the API that they promised, but it seems it is quite top-secret as no developers have heard a word since they announced the API in January. But it looks like you've got some pretty good information without it. :)

I plan on eventually installing my Raspberry Pi in the Volt to continuously log the data and wirelessly transmit it to my server for databasing when in range of either manually known or some generic public wifi, which should be pretty dang cool.

You might want to check out http://www.openvehicles.com/ -- they started off with a device for the Tesla that is similar to what you describe, and are now working on other EVs such as the Volt.