Streaming data in rust

on 2022-02-09

I wanted to attempt doing some programming puzzles to get more comfortable with rust. I had watched a great video of the solution in January.I was thinking as I was watching the video to solve the solution with more of a streaming oriented answer. I had been letting this percolate for a while until I was ambitious enough to try implementing the solution.

Input: Find the elf carrying the largest amount of calories. Where this is determined by the summing of values to the next empty line

4035
10596
17891
5278

11293
8478
10874
10582
10756
6649

I had the general steps on what I needed to do to accomplish this:

  1. Read the input file
  2. Parse each byte in the file and check for the newline character
  3. Group the items then parse them into an int
  4. Sum the container of ints when we hit an empty line and add this to another container.
  5. Find the largest value in the container for our answer

Read the file

let mut file = match File::open(filename) {
    Ok(file) => file,
    Err(error) => {
        println!("Error opening file: {}", error);
        return;
    }
};

Parse each byte

match file.read(&mut buffer) {
    Ok(0) => break,
    Ok(1) => {}

Group the items and parse into an int

let line_str = str::from_utf8(&line).unwrap();
let number = line_str.trim().parse::<i32>().unwrap();
numbers.push(number);
line.clear()

Sum the container of ints when we encounter an empty line.

if line.is_empty() {
    let total: i32 = numbers.iter().sum();
    totals.push(total);
    numbers.clear()

Max value of an interable.

println!("{:?}", totals.iter().max())

I would be curious to know if its better to split a file first then parse and sum it, versus the streaming method. However, we have been talking about stream processing data at work and thought it would be good practice to at least see how to put it together in theory.

Next step is to hopefully do something like this with some JSON input. I think that should be a little bit more challenging. Time to take a look at how these items are serialized. I know serde is supposed to be one of the best serialization projects, not only in rust, but accross all languages. I will have to take a read how they handle everything.

Full code

use std::fs::File;
use std::io::prelude::*;
use std::str;

fn main() {
    let filename = "input.txt";

    let mut file = match File::open(filename) {
        Ok(file) => file,
        Err(error) => {
            println!("Error opening file: {}", error);
            return;
        }
    };

    let mut buffer = [0; 1];
    let mut line = Vec::new();
    let mut numbers = Vec::new();
    let mut totals = Vec::new();

    loop {
        match file.read(&mut buffer) {
            Ok(0) => break,
            Ok(1) => {
                if buffer[0] == b'\n' {
                    if line.is_empty() {
                        let total: i32 = numbers.iter().sum();
                        totals.push(total);
                        numbers.clear()
                    } else {
                        let line_str = str::from_utf8(&line).unwrap();
                        let number = line_str.trim().parse::<i32>().unwrap();
                        numbers.push(number);
                        line.clear()
                    }
                } else {
                    line.push(buffer[0]);
                }
            }
            _ => panic!("Unexpected number of bytes read"),
        }
    }
    println!("{:?}", totals.iter().max())
}