What I’ve been working on
I’ve been working lately on a new Flutter app for mom-and-pop fast casual restaurants. The idea was brought to me when I was too early for a Bible study in the morning and the manager spoke with me about his app idea. Once he found out I was a software engineer, he could not resist the temptation to tell me about his app idea, but he’s only human!
So after a month and a half, him and I are now partnering on the app and it’s in closed alpha testing. Soon it will be released to his employees, and he can test it in his restaurant live. Now that we’ve hit our second iteration of testing in the alpha phase, I have some free time. With that free time, I will be analyzing my running data for an upcoming talk I am giving at https://tricities.dev.
Background
I’m training for an ultra marathon, a 50 mile race in May. In preparation for the marathon, I’ve been wearing a Garmin watch to track all of my runs. This has generated a lot of data. Analyzing this data would not only build skills that I’ve wanted to build, but would be fun to see when I run the best given a set of circumstances. Garmin data contains all kinds of juicy information such as sleep, weather, and other additional datapoints to accompany the running.
Julia
I’m using Julia for this data analysis. Julia as a language has a lot of features that I find to be compelling. Chief among those is the type system, broadcasting, and multiple dispatch. It’s simple to write code that acts on an entire collection at once by suffixing your function name with a dot.
do_a_thing.(collection) # Automatically breeze through the collection. Apply do_a_thing to all elements.
So, with a directory full of Garmin FIT files, I can use broadcasting and multiple dispatch to parse them all.
Garmin FIT Protocol
Garmin has their own open binary protocol that their devices store information in. There exist SDKs in most languages to read these files. Julia does not have an SDK for FIT files, so I will be building one!
The files have a header, and a data section.
Progress so far
My Garmin FIT.jl package can read the header of a FIT file! As it stands, it’s only reading sections of bytes from the file at a time and parsing those bytes.
The header of the FIT file, as expressed in a Julia struct:
struct FITHeader
sz::UInt8
protocol::UInt8
version::Integer
size::Integer
type::String
crc::AbstractVector{<:Real}
end
The most exciting work came from the version, size, and type members of the struct. Each of these members is represented as an array of bytes. Garmin describes version as 2 bytes, with the least significant byte being first, and the most being second. Garmin describes the type as an array of bytes representing an ASCII string (usually just “.FIT”). I could have kept these bytes as just a vector of UInt8 numbers, but I set their types to be something human readable to make them easier to work with in code. When reading a file, we would need to know the size of the file in bytes, not what the size of the file in bytes looks like as a byte string!
Enter the following functions, that both display some of the power of Julia:
"""
byte_vec_to_int(byte_vec::Vector{UInt8})::Real
FIT headers have a data size field, with 4 bytes.
"""
function byte_vec_to_int(byte_vec::Vector{UInt8})::Integer
result::Integer = 0
byte_vec = Int64.(byte_vec)
for i ∈ 0:length(byte_vec) - 1
result += byte_vec[1 + i] << (8 * i)
end
return result
end
"""
byte_vec_to_string(byte_vec::Vector{UInt8})::AbstractString
Given the byte vec, create a String
"""
function byte_vec_to_string(byte_vec::Vector{UInt8})::AbstractString
String(Char.(byte_vec))
end
byte_vec_to_int is my answer for converting Garmin’s byte arrays into integers. We start by passi9ng the byte_vec into to function, and converting all of the bytes into Int64. We then iterate over each of the bytes using the for loop (and the fancy ligature), summing the bytes in our running total. Within the running sum, we have to perform bitshifting to get the correct number. This is because of Garmin’s protocol describing the bytes in increasing order of significance. So, what does this mean?
num = 0x123456 # Each two numbers after the x is a byte. Bytes are 8 bits.
byte_vec::Vector{Uint8} = [0x56, 0x34, 0x12] # This is what the bytes look like coming out of the file, for the above number.
# To sum these bytes, we have to take into account their position in the original number.
# LSB: 0x56 == 0x000056
# +0x003400
# +0x120000
# ---------
# 0x123456
#
Note the trailing 0’s in the comment above. We get these trailing zeroes via the line within the loop:
for i ∈ 0:length(byte_vec) - 1
result += byte_vec[1 + i] << (8 * i) # Right here. Shift the number at 1 + i (Julia is 1-based) by 8 * i (8 bits in a byte).
end
The conversion of all numbers to an Int64, ensures us that we have enough space to the left of our byte to shfit the byte by 8*i. Otherwise, we would lose the bytes we are trying to shift as the “fall off” the left of the number.
byte_vec_to_string shows why Julia is an absolute powerhouse of a language. It uses broadcasting to cast all of the bytes to chars. ASCII characters are just bytes, which is a very simple encoding. Using `Char.`, we convert all bytes in the vector to Chars. We pass the resulting Char vector to the String function which collects the Chars in the vector and returns one String representation. The return type of AbstractString ensures that this works with functions that require any type of string, but that is out of scope of this post.
So there you have it! I’ll be reading the data section of the FIT files next, and after that I’ll be running an actual Pluto analysis on the data to give my talk at TriDev. Check in soon for more!