Converting RGB to Hex using Lodash

You have: a string that looks like this 'rgb(255, 0, 255)'

You want: the hex code for it '#ff00ff' as a string

You must use: Lodash

const rgbToHex = (rgb: string) => {
  const nums = _.words(rgb, /[0-9]+/g); // remove leading "rgb" and parens
  const hex = _.map(nums, (num: string) => {
    const as16 = _.parseInt(num).toString(16);
    return `${_.size(as16) === 1 ? '0' : ''}${as16}`;
  });
  return `#${hex.join('')}`;
};

I wrote this but ended up not needing it and didn’t have the heart to just delete it.

Hope it helps someone!

Book review: Grokking Continuous Delivery by Christie Wilson

Six-word summary: excellent tech-agnostic introduction to CI/CD
Quick rating: 5/5


Wow – even though I’m not a complete newbie to CI/CD, I think I learned something new every few pages. Or improved my understanding of something. Or questioned an assumption I’ve held for years.

Christie Wilson’s writing is conversational and pleasant, but that doesn’t mean the book is for kiddies – the 370 pages cover much more than just the basics. There’s loads of vocabulary, concepts, and fictional scenarios designed to highlight the incredible value each “step” of a CI/CD pipeline brings to a team.

I particularly enjoyed the “human element” woven throughout this book. Techniques and tools are introduced as solutions to specific problems familiar to developers and customers: painful middle-of-the-night calls, crunch around release, lengthy release cycles, long waits for features/bug fixes, and more. It’s not just a presentation of technologies, it’s a presentation of solutions.

See “Grokking Continuous Delivery” on Amazon.com

A sample page from early in the book.

Where I was starting from

Before reading this book, I would have described my understanding of CI/CD as “enough to get by on”.

I knew the gist of it as a result of working on teams using CI/CD: push a change, then the tests run on the build machine, and depending on the outcome, either merge the change into the codebase or fix what broke. The idea was to keep the codebase in an always-usable state. (Which was a huge step up from places I worked in the 00s, which were a mixed bag of manually kicking off builds, few automated tests, and a whole lot of “well it worked on my machine”.)

But I still felt like I only had a vague idea of how it all fit together. I wanted a better understanding of the underlying philosophies and goals. I also wanted to nail down some of the correct terminology for things (turns out it’s not “Jenkins is busted again”).

What’s inside

Length: 13 chapters, ~370 pages, lots of pics

Time to read: About 2 weeks of casual reading (a chapter every day or two)

Subjects covered: continuous integration/delivery/deployment, best practices, testing philosophies (unit vs integration vs e2e), “config-as-code”, typical pipeline structure, adding CI/CD to existing projects, dealing with flaky tests, test “noise”, some tech-specific stuff near the end (a quick tour of modern-day CD systems and version control systems), incremental feature development, dependency versioning, rollbacks, code coverage, shell scripting, parallelization…. and lots, lots more

I found the book surprisingly readable – this isn’t a dry tome of configuration or theory. The chapters flow nicely and build on previous ones. The diagrams, takeaway sections, and illustrations grabbed my attention in a way that just kinda pulled me through the pages.

Another peek inside the book.

What I’ve learned

A lot.

The most surprising thing I “learned” was how many of my assumptions about releasing software were laid bare and challenged by this book.

  • If releasing is painful, you should release more often.
  • An ideal deployment frequency is multiple times a day.
  • You don’t need to wait ’til the next new project to get started with CI/CD. Even if you can’t move all of a project to CI/CD, you can move parts of it – and doing so is valuable.
  • Feature work can (and should) be merged into develop incrementally. (Not to sound clickbait-y but this one really shocked me. I’m more familiar with mega-PRs that contain the entirety of a new feature and about a thousand merge conflicts.)

I think it helped, also, that I recognized so many of my “real life” work teams in this book. I have definitely been on teams that do some combination of the following:

  • tests that pass or fail randomly so we disable them
  • tests that take way too long to run so we run them sparingly
  • rolling out new features to everyone only to discover it’s broken and have to quickly unroll it out
  • deployments that are done manually by some poor sucker
  • processes/configs that live in someone’s head instead of in the codebase
  • delaying a release because we’re scared of what’s in it
  • merge a new feature only when it’s complete and includes 50+ changed files full of surprises and things that have to be updated and adapted to other in-progress work
  • more stuff I’ve tried to forget

I wish I could send this book back in time to those teams 😅

Who this book is for

If you’re already working in DevOps or you’ve set up some pipelines yourself, then you probably know most of the stuff this book covers. There are some general references to real-life technologies in this book, but this book is more about the concepts and goals of CI/CD, not a tutorial you can follow (although after reading it, I felt like I was ready to dig into the tutorials and docs).

If you are early in your career or your company is just starting to talk about CI/CD and all you know is the acronym, then this book will be an excellent introduction to continuous delivery.

If you have a few years of experience and know of CI/CD (and maybe you’ve worked on a team that uses it) then you’re probably going to get a lot out of this book. A few topics will probably be familiar, but the rest of it will expand your CI/CD knowledge and give you a much broader and more organized understanding of a wide variety of continuous delivery topics.

I could also see this book as being really useful to product owners, managers, and other people in roles that work adjacent to software engineers. You don’t need to know any programming languages to understand this book.

Why read a book? The docs are free!

This is just my personal preference, but I like being spoon-fed information by someone who has already organized their thinking on a topic. I didn’t know what docs to read, or in what order, and I wanted some understanding of the concepts before I tried to do anything myself.

Now I feel ready to dive into the docs and tools.

PS: If you buy the textbook, you also get access to ebook versions for free

I’m kind of a “dead tree” book person myself (they don’t compete for screen space, and I spend enough time looking at screens as it is), but the ebook version of Grokking Continuous Delivery was a pleasant surprise – there’s a nice table of contents on the left, it’s searchable, and the pics are all in color.

Book is available as pdf, epub, kindle, and liveBook
Here’s what the liveBook version looks like in Chrome – the table of contents down the left side is particularly pleasant.

The ebook came in handy for deciphering the one error I found in the book – on page 204, the “Top Dog Maps” architecture diagram overlaps some of the text.

On the left: the diagram overlaps a line of text in the book. On the right: the publisher has corrected the problem in the ebook and future print editions will contain the correction.

And that’s it – thanks for reading! And if you enjoyed this book (or want to recommend a similarly awesome one) please leave a comment!

See “Grokking Continuous Delivery” on Amazon.com

LeetCode 22: Generate Parentheses solution, TypeScript

This summer I took a break from side projects and gave LeetCode a try. It’s been nearly 3 years since my data structures and algorithms course with OSU, so I wanted to give myself a refresher on solving these kinds of problems.

This particular problem, “Generate Parenthesis“, was challenging for me at first but I revisited it a couple times over the summer and on the most recent try, I developed a solution that I think is fairly simple to understand, so I wanted to share it here. (I admit it’s not the most performant solution, but it passes.)

The premise

Given n pairs of parentheses, write a function to generate all combinations of well-formed parentheses.

Example 1:

Input: n = 3
Output: ["((()))","(()())","(())()","()(())","()()()"]

Example 2:

Input: n = 1
Output: ["()"]

Constraints:

1 <= n <= 8

My solution

I started this problem by writing out what would be generated when n =1, when n = 2, when n = 3, and when n = 4. This step helped me see what I needed to generate and it gave me an opportunity to spot patterns.

When n = 0
result = [""]

When n = 1
result = ["()"]

When n = 2
result = ["()(), (())"]

When n = 3
result = ["()()()","((()))","()(())","(())()"]

When n = 4
result = ["()()()()","(((())))","(()(()))","((())())","()((()))","((()))()","()()(())","()(())()","(())()()"]

The pattern:

I noticed that every time I generated a new result, I was building on the previous n value’s result array.

Specifically, the steps are:

  1. Take each result in the previous n value’s array and wrap it in parentheses. Each of those “wrapped” results becomes part of the new n value’s results array.
  2. Take each result in the previous n value’s results array and insert a new “()” at each possible location in the string. Each of those “inserted” results becomes part of the new n value’s results array.

You might realize at this point that some duplicate results will be generated during this process. This is why I put the results into a Set (so that duplicates are ignored).

Here is this approach, written in TypeScript:

function generateParenthesis(n: number): string[] {
    if (n === 1) { return ["()"]; }
    
    let result = new Set<string>();
    let previousCombos = generateParenthesis(n-1);
    
    for (let i in previousCombos) {
        let combo = previousCombos[i];
        
        // now step through each one and embed a new paren at each possible position
        for (let i = 0; i < combo.length; i++) {
            result.add(combo.substring(0,i) + '()' + combo.substring(i));
        }
    }
    
    return [...result];
};

View as a gist on Github

Runtime analysis

Honestly, I don’t fully understand the runtime on this one, since it’s recursive and since each subsequent step draws from a growing list of “previous step” solutions. It “feels” exponential, but it’s not quite – or at least, the quantity of results in the sets that I worked out by hand is not increasing exponentially.

Pressed to come up with something, I’d say it’s O(2n) because:

  • We have to “handle” every 1…n (that’s the first n)
  • And every time we “handle” an n, we have to produce n * x results, where x is the length of the previous n’s results and that length grows (considerably) with each step.

From this article, “The growth curve of an O(2n) function is exponential – starting off very shallow, then rising meteorically.”

Hey, that sounds like this problem… right?

(If you know how to figure this one out with greater accuracy I’d love to hear it in the comments.)

Performance

[Update] 1 year later: a dynamic programming solution

November 2022 – I showed this blog to a friend and he developed a neat solution that builds a 2D array where each index holds all of the previous step’s calculations. What’s clever about this solution is that it does not generate duplicates the way my Set-based solution above does.

function nextPermutation(previousPermutations: string[][]): string[] {
    const nextPermutationsLength = previousPermutations.length;
    let nextPermutations = [];
    for (let i = 0; i < nextPermutationsLength; i++) {
        // each of the previous permutations becomes our "inner" permutations (array)
        const innerPermutations = previousPermutations[i];
        
        // calculate how many outer permutations we need based on where we are in the loop
        const outerPermutationsLength = nextPermutationsLength - 1 - i; 
        
        // grab from that position to get "outer permutations"
        const outerPermutations = previousPermutations[outerPermutationsLength];
        
        //  now that we have the outer permutations, step through them...
        let newPermutations = outerPermutations.flatMap(outer => innerPermutations.map(inner => `(${inner})${outer}`));
        newPermutations.forEach((newPerm) => {
            nextPermutations.push(newPerm);
        })
    }
    
    return nextPermutations;
}

function generateParenthesis(n: number): string[] {
    let result = [[""]];
    
    for (let i = 1; i < n+1; i++) {
        // push the array of new permutations into the result array 
        result.push(nextPermutation(result));
    }
    
    // result[n] now holds the array we need... 
    return result[n];
};

View as a Gist on Github

Leetcode 1101: The Earliest Moment When Everyone Become Friends solution, TypeScript

The premise:

In a social group, there are n people, with unique integer ids from 0 to n-1.

We have a list of logs, where each logs[i] = [timestamp, id_A, id_B] contains a non-negative integer timestamp, and the ids of two different people.

Each log represents the time in which two different people became friends.  Friendship is symmetric: if A is friends with B, then B is friends with A.

Let’s say that person A is acquainted with person B if A is friends with B, or A is a friend of someone acquainted with B.

Return the earliest time for which every person became acquainted with every other person. Return -1 if there is no such earliest time.

Example test case:

Input: logs = [[20190101,0,1],[20190104,3,4],[20190107,2,3],[20190211,1,5],[20190224,2,4],[20190301,0,3],[20190312,1,2],[20190322,4,5]], n = 6
Output: 20190301

My solution, in Typescript:

function earliestAcq(logs: number[][], n: number): number {

    // sort the logs by timestamp (earliest first)
    logs.sort((a:number[], b:number[]) => {
       return (a[0] - b[0]);
    });

    // this map is going to serve as a container for all the sets
    let mapOfSets = new Map();
    
    // put every friend ID into its own individual set
    for (let i = 0; i < n; i++) {
        let individualSet = new Set();
        individualSet.add(i);
        mapOfSets.set(i, individualSet);
    }
    
    // now every friend is its own set in the map 
    // console.log(mapOfSets);
    
    // step through the logs and join the sets as matches are encountered 
    for (let i = 0; i < logs.length; i++) {
        let friendA = logs[i][1];
        let friendB = logs[i][2];

        // friendA and friendB have "met" - get their sets
        
        let setA = mapOfSets.get(friendA);
        let setB = mapOfSets.get(friendB);
        
        // and if their sets are not equal...
        if (setA != setB) {
            // join the two sets by adding all of the friends from setB to setA
            for (let friend of setB) {
                // add all the friends from setB to setA
                setA.add(friend);
                
                // update map of sets such that setA is pointed at by each friend originally from B
                mapOfSets.set(friend, setA);
            }
            
            if (setA.size === n) {
                // every friend is now accounted for in set A
                // return the timestamp where this was achieved
                return logs[i][0];
            }
        }
    }
    
    return -1;
};

View as a gist on GitHub

My solution, explained:

  1. Sort all the inputs by timestamp so the earliest ones are first
  2. Create a new map and fill it with sets representing the currently known friendships. The keys will be the friend “ID numbers”, and the values will be the sets representing who that friend is friends with. In the beginning, a friend is only friends with themselves. mapOfSets = {0:{0}, 1:{1}, 2:{2}, 3:{3}, 4:{4}, 5:{5}}
  3. Now, step through the logs and grab each friend pair (friendA and friendB) from that log.
  4. Get the set associated with friendA, and the set associated with friendB, and see if they’re identical
  5. If the sets are not identical, copy all the friends from setB to setA and for each of those relocated friends, update their own value in the mapOfSets to point at setA
  6. If setA becomes equal in size to n (the number of friends), then this is the moment where “everyone knows everyone else” and we can return the timestamp of the current log

Runtime analysis:

O(n log n) due to the sorting

Performance:

Ehh, I doubt this was the absolute best way to do this – it’s just because so few other Typescript submissions exist.

Building a Flutter app: re-ordering lists

This post is part of my “Building a Flutter App” dev journal, which begins here.

In this post: I develop a feature that allows a user to manually re-order a shopping list. Taking inspiration from the way Spotify lets users re-order a playlist, Grocery Go’s users should be able to put the items in any order they like.

First, the final product

You can view this step’s pull request here.

This gif demonstrates the feature. The user re-orders items in a list and changes which “store” is active to set different item orders for different stores.

Getting set up – where we’re starting from

When I started work on this feature, the user could already create a shopping list and populate it with items, like so:

However, there is no way to sort or re-order those items.

As a user, I think it’d be useful if the items could be re-ordered to match the order I actually pick them up in when I’m at the store.

But which store? I shop at many different stores, and they’re all laid out differently, so in addition to creating a default order I also want to be able to create variations on the default order and “save” those variations to each store that this list applies to.

Planning the work: UI inspiration

For inspiration, I looked at how Spotify’s phone app (or at least iOS app) gives the user the ability to reorder items in a list.

To change the order of items in this playlist, the user taps the “three dots” adjacent to the “down arrow” above the “add songs” button.

A separate screen pops up and offers the option to “Edit” the playlist:

Tapping “Edit” opens up a modal in which the individual songs can be pressed on and dragged up and down in the list.

In this screenshot, I am dragging the song named “Passage” up in the list. It follows my finger as long as I keep pressing the screen.

Feature list

  • Every shopping list has a “default” order
  • Every shopping list can be linked with 1 or more stores, and each of those store links has its own item order (initially copied from ‘default’)
  • The user can re-order items in the “default” list and the store-specific lists
  • The user can change which store list is being displayed via the shopping list view
  • Creating a brand new item adds it to the end of the “default” list as well as to the end of all store-specific lists

Modeling the data

The user can store different item sequences for different stores, like so:

Default – items are, by default, shown in the order they were created for this list. The user can re-order this list, though. Newly created items are added to the bottom (end) of this list.

  • Party size bag of M&Ms
  • Bread
  • Bananas
  • Milk
  • Eggs

Safeway – the user wants to see the items in this order when they are at Safeway. Newly created items are added at the bottom of this list.

  • Bananas
  • Bread
  • Milk
  • Eggs
  • Party size bag of M&Ms

In the Firebase data structure, I imagined each item would have a map of key/value pairs where the key is the store’s ID and the value is the position in that store’s list.

This worked well and I duplicated this structure for the shopping lists themselves, allowing the shopping lists to be re-ordered on the main screen in addition to the items in each list.

Trouble with “orderBy” and Streams

Initially, I tried to get the items in order (for whatever store list was selected) like this:

Stream<QuerySnapshot> getItemsStream(shoppingListID, isCrossedOff, storeID) {
    return shoppingLists.document(shoppingListID).collection('items').where('isCrossedOff', isEqualTo: isCrossedOff).orderBy('listPositions.$storeID').snapshots();
  }

I thought this was a very clever and the “quite obvious” approach, but the more I tested it, the more apparent it became that using “orderBy” made it so the widget(s) displaying the contents of that stream wouldn’t redraw in the UI, even if coming back from another route. It also broke the ability to cross off items: they would appear in the active and inactive lists at the same time, or they would appear in neither list, until the user reloaded that page of the app.

I went down a lot of different roads trying to fix this, but ultimately all I did was stop using .orderBy and sorted everything on the front-end (in-widget) instead. I don’t know if that’ll end up being a bad idea, but it was the only way I could get both streams and data ordered by some criteria to work together.

Sorting the Stream results

The stream-getting methods in database_manager.dart return a Stream of QuerySnapshots, like so:

Stream<QuerySnapshot> getActiveItemsStream(shoppingListID, storeID) {
    return shoppingLists.document(shoppingListID).collection('items')
        .where('isCrossedOff', isEqualTo: false)
        .snapshots();
  }	  }

And then over here in main_shopping_list.dart, I set a state variable (activeItemsStream) to the return value of that “get stream” call:

void initState() {
    super.initState();
    getSharedPrefs().then((storeIDFromPrefs) {
      activeItemsStream = db.getActiveItemsStream(widget.list.id, storeIDFromPrefs); // should get selectedStoreID from state
      inactiveItemsStream = db.getInactiveItemsStream(widget.list.id, storeIDFromPrefs);
    }); // sets selectedStoreID
  }

To display it, I used a custom widget that I made myself called ItemListStream that takes that state variable as a parameter (called dbStream, first one in the list) and a sortBy parameter.

ItemListStream(dbStream: activeItemsStream, sortBy: selectedStoreID, listType: 'item', onTap: _updateCrossedOffStatus, onInfoTap: _editItem, parentList: widget.list),

That widget file is actually pretty short, here is item_list_stream.dart in its entirety. Notice the sort performed on the list items, this takes the place of “orderBy” on the database call.

import 'package:cloud_firestore/cloud_firestore.dart';
import 'package:flutter/material.dart';
import 'package:grocery_go/components/item_list.dart';
import 'package:grocery_go/models/shopping_list.dart';

class ItemListStream extends StatelessWidget {

  final dbStream;
  final sortBy;
  final listType; // item, crossedOff
  final onTap;
  final onInfoTap;
  final ShoppingList parentList;

  ItemListStream({@required this.dbStream, @required this.sortBy, @required this.listType, @required this.onTap, @required this.onInfoTap, this.parentList});

  @override
  Widget build(BuildContext context) {
    return StreamBuilder(
      stream: dbStream,
      builder: (context, snapshot) {
        if (snapshot.hasError) {
          return Text('Error: ${snapshot.error}');
        }
        if (snapshot.hasData && !snapshot.data.documents.isEmpty) {
          List<DocumentSnapshot> docs = snapshot.data.documents;

          if (listType != 'crossedOff') {
            docs.sort((a, b) {
              return a.data['listPositions'][sortBy].compareTo(b.data['listPositions'][sortBy]);
            });
          } else {
            docs.sort((b, a) {
              return a.data['lastUpdated'].toDate().compareTo(b.data['lastUpdated'].toDate());
            });
          }

          return ItemList(list: docs, listType: listType, onItemTap: onTap, onInfoTap: onInfoTap, parentList: parentList);
        } else {
          return Column(
              children: [
                Padding(
                  padding: EdgeInsets.all(8),
                  child: Text("No items yet!"),
                ),
              ],
          );
        }
      }
    );
  }
}

Building the “reorder” UI

I tap the “three dots in a row” in the blue “Items” bar, then I drag “Bag of potatoes” up one spot. The change is reflected on the previous screen and in Firebase.

Check out reorderable_list.dart to see how this works.

I adapted the reorder logic from this very helpful example: https://gist.github.com/slightfoot/bfaaf6338d85e27b2acfe1b265ee5f27

Building the “store change” UI

Finally, the user needed a way to change which store was selected. Here’s what I built:

I built this using a Cupertino Action Sheet. Look in main_shopping_list.dart for the full code. Changing the selected store calls _setSelectedStore, which updates the selectedStoreID state variable and “re-gets” the active items and inactive items streams with that updated ID.

_setSelectedStore(String id, String name) async {
    final SharedPreferences prefs = await SharedPreferences.getInstance();
    prefs.setString(widget.list.id, id);

    setState(() {
      selectedStoreID = id;
      selectedStoreName = _getStoreName(id);
      activeItemsStream = db.getActiveItemsStream(widget.list.id, id);
      inactiveItemsStream = db.getInactiveItemsStream(widget.list.id, id);
    });

    Navigator.pop(context, id);
  }

The final product

And that’s it for this feature! Here’s the feature’s pull request.

Here’s what we did:

  • Added a new view that lets the user change the order of shopping list items by dragging and dropping them individually
  • Added the ability to change which store is active for a shopping list
  • Added “listPositions” map to each item so each item knows where it appears in each store

Go back to Part 7’s feature work list.

Building a Flutter app: creating a many-to-many relationship between stores and shopping lists

This post is part of my “Building a Flutter App” dev journal, which begins here.

In this post: I develop a feature that allows the user to “link” stores and shopping lists to each other. This link is managed on the “edit store” and “edit shopping list” pages. A link is bidirectional: adding a store to a shopping list also adds that shopping list to the store.

First, the final product

The user can toggle every one of their stores on/off for each shopping list.

Feature description and purpose

This feature has a few purposes:

  • allow items from multiple shopping lists to appear in one store (so when you’re at, for example, “Fred Meyer”, you can see items for “groceries” and “home improvement”)
  • allow the user to save different item orders for different stores (this work was documented in this article)
  • make it easier for the user to know which store they need to go sooner to based on which items they need

Deciding how to represent the store/list links in the database

First, I consulted Firebase’s docs on structuring data to see what they had to say about representing this kind of many-to-many relationship. Their “users and groups” example is very close to what I want to build here.

They recommend using an index of groups, like so:

// An index to track Ada's memberships
{
  "users": {
    "alovelace": {
      "name": "Ada Lovelace",
      // Index Ada's groups in her profile
      "groups": {
         // the value here doesn't matter, just that the key exists
         "techpioneers": true,
         "womentechmakers": true
      }
    },
    ...
  },
  "groups": {
    "techpioneers": {
      "name": "Historical Tech Pioneers",
      "members": {
        "alovelace": true,
        "ghopper": true,
        "eclarke": true
      }
    },
    ...
  }
}

(This was taken directly from the Firebase docs)

Next, I adapted their example to match my project. This is just mock data in a text file, it’s not in the project’s codebase anywhere.

// An index to track a shopping list's stores
{
  "shopping_lists": {
    "list123": {
      "name": "Groceries",
      "stores": {
         "store890": true,
         "store567": true
      }
    },
    ...
  },
  "stores": {
    "store890": {
      "name": "Safeway, Kirkland",
      "shopping_lists": {
        "list123": true,
        "list124": true
      }
    },
    ...
  }
}

Now I have a clear goal to work towards, but this should be straightforward to implement. I will have to update/maintain the link data in two places, so my Database Manager functions will account for that.

Inserting the data into the database by hand

My data differs from Firebase’s sample data in that I use the auto-generated IDs to identify my records, but I don’t show the user those IDs, I show them the name instead. I decided to change my mock data to look like this, instead:

  "shopping_lists": {
    "list123": {
      "name": "Groceries",
      "stores": {
         "store890": "Safeway, Kirkland",
         "store567": "Fred Meyer, Kirkland"
      }
    },
    ...
  },

Now the store ID are the key and store name is the value.

The next step was to put this data into my Firebase document by hand. Here is my “Back to school stuff” shopping list with two stores associated with it.

Now I’ll have something to display in the UI (and later edit).

Building the store/list linking UI

The first page I worked on was the “Edit shopping list” page. I considered doing a list of toggle switches under the “List name” field, but that list could potentially be very long and I didn’t want to push the “Save” button off screen.

Instead, I decided to show a short (truncated) list of linked stores on this page and provide a link that opens up a new view full of toggle switches that represent each possible store link.

Note: I knew that whatever UI I built for this feature would also be used by the “Edit Store” page, so I built everything using (reusable) components and named them generically.

shopping_list_form.dart now contains an internally-used method titled _formFields. This method builds a List of widgets and includes _nameField() in it by default. Whether you’re editing or creating a new shopping list, you’ll always get _nameField.

If the shopping list id is not null, then we can also show the “linked entities”.

 _formFields() {
    List<Widget> fields = [_nameField()];
    // if we're editing an existing shopping list, add the linked stores
    if (widget.shoppingList?.id != null) {
      fields.add(_linkedEntities());
    }

    return Container(
      child: Column(
        crossAxisAlignment: CrossAxisAlignment.start,
        children: fields,
      ),
    );
  }

_linkedEntities is a separate method that returns another widget, LinkedEntitiesList.

  _linkedEntities() {
    return LinkedEntitiesList(
        widget.shoppingList.id, "shopping list", widget.shoppingList.name, widget.shoppingList.stores, "Stores");
  }

linked_entities_list.dart is shown here in its entirety from the final version of the feature branch. The thing that’s interesting here is the use of the spread operator to build an “entity list” out of some unknown number of list elements. This turned out to be a good technique for adding a variable number of children to a column’s children array.

import 'package:flutter/material.dart';
import 'package:grocery_go/views/manage_links.dart';
import '../../db/database_manager.dart';

class LinkedEntitiesList extends StatelessWidget {
  final String parentID;
  final String parentName;
  final String listType;
  final Map linkedEntities;
  final String entities;

  LinkedEntitiesList(this.parentID, this.listType, this.parentName, this.linkedEntities, this.entities);

  final DatabaseManager db = DatabaseManager();

  @override
  Widget build(BuildContext context) {

    _goToManageLinks() {
      var stream;
      if (listType == "shopping list") {
        stream = db.getStoresStream();
      } else if (listType == "store") {
        stream = db.getShoppingListStream();
      } else {
        print("Error: unrecognized list type in linked_entities_list.dart");
      }

      Navigator.pushNamed(context, ManageLinks.routeName, arguments: ManageLinksArguments(dbStream: stream, linkedEntities: linkedEntities, parentID: parentID, parentName: parentName, parentType: listType));
    }

    var _list = linkedEntities != null ? linkedEntities.values.toList() : [];

    return Container(
      height:300,
      child: Column(
        crossAxisAlignment: CrossAxisAlignment.start,
        children: <Widget>[
          _listTitle(),
          ..._entityList(_list),
          _manageLinksButton(_goToManageLinks),
        ],
      ));
    }

  _listTitle() {
    return Text("$entities", style: TextStyle(fontSize:16, fontWeight: FontWeight.bold));
  }

  _entityList(list) {
    const MAX_LIST_LEN = 4;

    if (list == null || list?.length == 0) {
      return [Text("This $listType is not attached to any $entities yet.")];
    } else {
      var listLen = list.length > MAX_LIST_LEN ? MAX_LIST_LEN : list.length;
      var entityList = List();
      for (var i = 0; i < listLen; i++) {
        entityList.add(Text(list[i].toString()));
      }
      if (list.length > MAX_LIST_LEN) {
        entityList.add(Text('+${list.length - MAX_LIST_LEN} more'));
      }
      return entityList;
    }
  }

  _manageLinksButton(onPressedAction) {

    return FlatButton(
      onPressed: () => onPressedAction(),
      child: Text("Add/Remove $entities"),
      textColor:Colors.blue,
      padding: EdgeInsets.all(0),
    );
  }
}

Notes:

  • I called them “entities” here because these widgets work for stores or shopping lists
  • I had to make a similar set of changes to the store_form.dart component
  • Getting a Column widget to have different children based on some condition was a new challenge. The best solution I could come up with was to use a List, called “fields” in this example, to hold the widgets that should be the column’s children. If a condition is met (in my case, shopping list id is not null), then a widget is pushed to the fields List (otherwise, it is not pushed). That approach seemed to be a good way to make the Columns children vary as needed.
  • user2875289‘s answer in this Stack Overflow question (method 2 and method 3 to be exact) helped me figure out how to iterate through a list to create Text widgets and how to include another Text widget child in that same array of children.

Firebase updates: adding a new store (or shopping list) to the map

Every Shopping List is going to have a map of Stores, like this:

"shoppingListID01" : {
  "name": "Groceries",
  "stores" : {
    "storeID01": "Safeway",
    "storeID02": "Fred Meyer"
  }
}

And when a new store is added, it has to be added without affecting the rest of the. map:

"stores" : {
  "storeID01": "Safeway",
  "storeID02": "Fred Meyer",
  "storeID03": "Home Depot" // NEW ONE, didn't mess up Safeway or Fred Meyer
}

And vice versa for Stores – every Store record keeps a map of its linked Shopping Lists.

"storeID01" : {
  "name": "Safeway",
  "stores" : {
    "shoppingListID01": "Groceries",
    "shoppingListID02": "Pool party stuff"
  }
}

Every “link” is created in two places:

  • Adding a store to a shopping list also adds that shopping list to that store
  • Adding a shopping list to a store also adds that store to that shopping list

For the sake of “Step 1” here I am just going to work on adding a new store link to the map of stores.

My initial (dead end) approach with “update” and “merge: true” and why that didn’t work with Cloud Firestore

From reading the Firebase docs I knew I was going to need to use update and {merge: true} because I want to keep the rest of the object untouched.

This was my first attempt, but it doesn’t work because apparently “update” is not usable if you’re using the cloud firestore package.

Future updateShoppingListLink(String parentStoreID, String entityID, bool val) async {

    DocumentReference storeRef =  stores.document(parentStoreID);

    Map<String, Map> data = {
      "shoppingLists": {
        entityID: val
      },
    };

    storeRef.update(data, {merge: true}); // doesn't work, "update" doesn't exist 
  }
}

I looked more closely at the cloud firestore docs, and they show “updateData” instead of “update”, so I swapped update for updateData:

storeRef.updateData(data, {merge: true}); // doesn't work, doesn't accept "merge"

updateData doesn’t accept the “merge” object, though. It says “too many positional arguments”.

Just to see what would happen, I took off the merge object and updated my database entry without the merge flag present.

storeRef.updateData(data);

The good news was – this “worked” in the sense that the new shopping list ID was successfully added to the store document:

… but as soon as you add a second store, that first store was overwritten and replaced with the new one:

At this point, I figured I was just not correctly passing “merge true” to updateData. I found this seemingly-related thread from 2017 that referenced this merged code that suggested SetOptions.merge() could be applied.

Here’s their example (I took this from their test, I couldn’t find it in their docs):

test('merge set', () async {
        await collectionReference
            .document('bar')
            .setData(<String, String>{'bazKey': 'quxValue'}, SetOptions.merge);
        expect(SetOptions.merge, isNotNull);

Which I adapted to my code:

storeRef.updateData(data, SetOptions.merge); // doesn't work, expects 1 parameter not 2

Making it all one object didn’t work, either:

storeRef.updateData({data, SetOptions.merge}); // also does not work

Then I found this downvoted Stack Overflow reply suggesting this syntax, which also does not work (it wants 1 argument, not 2):

storeRef.updateData(data, SetOptions(merge: true)); // also does not work

I kept digging and found this thread on the issue, which was last updated less than two months ago and says that merge and mergeFields are coming to the FlutterFire plugin set (which includes the cloud firestore plugin) in this update. Specifically, here are the “patch notes” for the upcoming changes to cloud firestore, which are still in review as of this writing (July 2020).

Ahh, so it seems there’s a big update coming soon that will fix this, but “merge true” is a lost cause at this point in time. If you’re reading this in the future, perhaps you have the updated cloud firestore package and none of this is a problem, but for those of us using it as it is now, I thought I’d see if I could find a workaround.

I started to wonder if I could just access shoppingLists with dot notation and gave this a try:

// working example of how to update one field in an existing map without deleting the others
  
Future updateShoppingListLink(String parentStoreID, String entityID, bool val) async {
    DocumentReference storeRef =  stores.document(parentStoreID);
    storeRef.updateData({'shoppingLists.$entityID': val});
  }

Yay, it worked!

This technique makes it possible to add a new field to the map without deleting the existing ones in the process.

Perhaps I didn’t need “merge true” in the first place, but I’ll leave my notes up in case they’re helpful to anyone else trying to update one entry in a map in a Firebase document.

TL;DR: “dot notation” was a good way to update specific field in a map contained within a Firebase document.

storeRef.updateData({'shoppingLists.$entityID': val});

Passing the store name as the value

Currently, the “val” passed to the shoppingList map is a boolean value but what I really need is the store’s name.

I changed the database_manager.dart method to take a String called name instead:

  Future updateShoppingListLink(String parentStoreID, String entityID, String name) async {
    DocumentReference storeRef =  stores.document(parentStoreID);
    storeRef.updateData({'shoppingLists.$entityID': name});
  }

And then I changed the toggleItem method in toggle_list.dart to pass the name instead of the value:

class _ToggleListState extends State<ToggleList> {

  final DatabaseManager db = DatabaseManager();

  toggleItem(entityID, entityName) {
    print(widget.parentType);
    if (widget.parentType == "shopping list") {
      db.updateStoreLink(widget.parentID, entityID, entityName);
    } else if (widget.parentType == "store") {
      db.updateShoppingListLink(widget.parentID, entityID, entityName);
    }
  }

  @override
  Widget build(BuildContext context) {

    return ListView.builder(
        shrinkWrap: true, // gives it a size
        itemCount: widget.list.length,
        itemBuilder: (BuildContext context, int index) {
          var item = LinkedEntity(widget.list[index]);

          return SwitchListTile(
            title: Text(item.name),
            value: widget.linkedEntities?.containsKey(item.id) ?? false,
            onChanged: (bool value) => toggleItem(item.id, item.name),
          );
        }
    );
  }
}

This works – hooray! – but it reveals a new problem: what happens when the user changes the shopping list’s name?

Thoughts on what happens when a shoppingList (or Store) gets renamed:

Firebase doesn’t seem to shy away from redundant data, and the alternative seems to be to store the IDs alone and then perform a “what name goes with this ID?” look up every time the user views or manages the linked lists (or stores).

In my app, it’s probably way more common to view/manage the links than it is to rename a shopping list or store, and I don’t anticipate users having more than about 5-10 of each, so I am going to (cautiously) proceed with the idea that it’s better to update the name in multiple places if the name changes vs. the idea that the name should be looked up every time the user views a list.

I’ll revisit what happens when a shopping list or store is renamed later on in this article.

Removing an existing link from the map

Everything I’ve done so far is for creating a link.

The user can also remove a link (by toggling it to “false” in the list), which I imagined would have a database equivalent of removing the item from the map entirely. What I don’t want to do is make a copy of the entire linkings map, remove the single entry that’s going away, and then push the entire updated map.

As usual, I began with a bit of research and found that the “dot notation” that served me so well for adding a field to a map can also be used with FieldValue.delete().

Here, I’ve written a ternary that looks at the value of val. If true, it updates the ‘stores’ or ‘shoppingLists’ map with the entity’s ID and name. If false, it removes the given entity ID from ‘stores’ or ‘shoppingLists’.

  Future updateStoreLink(String parentListID, String entityID, String name, bool val) async {
    DocumentReference shoppingListRef = shoppingLists.document(parentListID);
    val == true ? shoppingListRef.updateData({'stores.$entityID': name}) : shoppingListRef.updateData({'stores.$entityID': FieldValue.delete()});
  }

  Future updateShoppingListLink(String parentStoreID, String entityID, String name, bool val) async {
    DocumentReference storeRef =  stores.document(parentStoreID);
    val == true ? storeRef.updateData({'shoppingLists.$entityID': name}) : storeRef.updateData({'shoppingLists.$entityID': FieldValue.delete()});
  }

In toggle_list.dart I had to make a few changes to the toggleItem method. If widget.linkedEntities is null, it creates an empty new Map(). Without this, a store (or shopping list) that doesn’t have anything in its linked shoppingLists (or stores) map will be interpreted as ‘null’, causing .containsKey to throw an exception.

Here is toggle_list.dart in its entirety.

import 'package:flutter/material.dart';
import 'package:cloud_firestore/cloud_firestore.dart';
import 'package:grocery_go/db/database_manager.dart';

class LinkedEntity {
  String id;
  String name;

  LinkedEntity(DocumentSnapshot document) {
    this.id = document['id'];
    this.name = document['name'];
  }
}

class ToggleList extends StatefulWidget {

  final String parentType;
  final String parentID;
  final List list;
  Map linkedEntities;

  ToggleList({Key key, @required this.parentType, @required this.parentID, @required this.list, @required this.linkedEntities});

  @override
  _ToggleListState createState() => _ToggleListState();
}

class _ToggleListState extends State<ToggleList> {

  final DatabaseManager db = DatabaseManager();

  toggleItem(entityID, entityName, value) {

    if (widget.linkedEntities == null) {
        widget.linkedEntities = Map();
    }

    // update "locally"
    if (widget.linkedEntities.containsKey(entityID)) {
      setState(() {
        widget.linkedEntities.remove(entityID);
      });
    } else {
      setState(() {
        widget.linkedEntities[entityID] = entityName;
      });
    }
    // update in database
    if (widget.parentType == "shopping list") {
      db.updateStoreLink(widget.parentID, entityID, entityName, value);
    } else if (widget.parentType == "store") {
      db.updateShoppingListLink(widget.parentID, entityID, entityName, value);
    }
  }

  @override
  Widget build(BuildContext context) {

    return ListView.builder(
        shrinkWrap: true, // gives it a size
        itemCount: widget.list.length,
        itemBuilder: (BuildContext context, int index) {
          var item = LinkedEntity(widget.list[index]);

          return SwitchListTile(
            title: Text(item.name),
            value: widget.linkedEntities?.containsKey(item.id) ?? false,
            onChanged: (bool value) => toggleItem(item.id, item.name, value),
          );
        }
    );
  }
}

Here’s where we’re at now:

  • Stores and Shopping Lists have a map of their “linked entities” (stores keep track of their linked shopping lists, shopping lists keep track of their linked stores)
  • Linked entities can be linked/unlinked using a toggle switch
  • The data is changed in the database as the user toggles a link on/off

Making the link a two-way link

If the user adds the “swim stuff” list to the “Toys R Us” store, then “swim stuff” list should also get an association with the “Toys R Us” store. In other words, adding or removing a store from a shopping list should also add or remove that same shopping list from that same store.

Each existing method basically had to repeat the code of the other – once I wrote this out, I realized I could combine them into one method.

Future updateStoreLink(String shoppingListID, String storeID, String name, bool val) async {
    // add a store to the specified shopping list
    DocumentReference shoppingListRef = shoppingLists.document(shoppingListID);
    val == true ? shoppingListRef.updateData({'stores.$storeID': name}) : shoppingListRef.updateData({'stores.$storeID': FieldValue.delete()});

    // do the opposite - add this shopping list to the specified store
    DocumentReference storeRef =  stores.document(storeID);
    val == true ? storeRef.updateData({'shoppingLists.$shoppingListID': 'temp'}) : storeRef.updateData({'shoppingLists.$shoppingListID': FieldValue.delete()});
  }

  Future updateShoppingListLink(String storeID, String shoppingListID, String name, bool val) async {
    // add a shopping list to the specified store
    DocumentReference storeRef =  stores.document(storeID);
    val == true ? storeRef.updateData({'shoppingLists.$shoppingListID': name}) : storeRef.updateData({'shoppingLists.$shoppingListID': FieldValue.delete()});

    // do the opposite - add this store to the specified shopping list
    DocumentReference shoppingListRef = shoppingLists.document(shoppingListID);
    val == true ? shoppingListRef.updateData({'stores.$storeID': 'temp'}) : shoppingListRef.updateData({'stores.$storeID': FieldValue.delete()});
  }

Here they are refactored into one method, with a change made to the signature to take both the shopping list name and store name.

Future updateStoreShoppingListLink(String shoppingListID, String storeID, String shoppingListName, String storeName, bool val) async {
  // add this store to the specified shopping list
  DocumentReference shoppingListRef = shoppingLists.document(shoppingListID);
  val == true ? shoppingListRef.updateData({'stores.$storeID': storeName}) : shoppingListRef.updateData({'stores.$storeID': FieldValue.delete()});

  // and add this shopping list to the specified store
  DocumentReference storeRef =  stores.document(storeID);
  val == true ? storeRef.updateData({'shoppingLists.$shoppingListID': shoppingListName}) : storeRef.updateData({'shoppingLists.$shoppingListID': FieldValue.delete()});
}

Back in toggle_list.dart, I still have to call db.updateStoreShoppingListLink(...); in two different places because the concept of ‘parentID’ and ‘entityID’ are variable based on whether the user came in from the “edit store” flow or the “edit shopping list” flow.

When the user came in from “edit shopping list”, the parentID is a shoppingList’s ID. When the user comes in from “edit store”, the parentID is a store’s ID.

The updateStoreShoppingListLink method always expects the params in this order:

method params: (shoppingListID, storeID, shoppingListName, storeName, value)

… so we change the order of widget.parentID and entityID as dictated by the list type.

// update in database
// method params: (shoppingListID, storeID, shoppingListName, storeName, value)

if (widget.parentType == "shopping list") {
  // if we're editing a shopping list then the parent ID is the list ID and the entity is the store
  db.updateStoreShoppingListLink(widget.parentID, entityID, widget.parentName, entityName, value);
} else if (widget.parentType == "store") {
  // if we're editing a store, then the parent ID is the store ID and the entity is the shopping list
  db.updateStoreShoppingListLink(entityID, widget.parentID, entityName, widget.parentName, value);
}

In this demo, toggling the “Swim stuff” list on for the “Toys R Us” store also adds the “Toys R Us” store to the “swim stuff” list.

All of these use cases work now:

  • Add a shopping list to a store adds that same store to that shopping list
  • Remove a shopping list from a store removes that same store from the shopping list
  • Add a store to a shopping list adds that same shopping list to that store
  • Remove a store from a shopping list removes that same shopping list from the store
  • User can remove all the shopping lists from a store
  • User can remove all the stores from a shopping list
  • Create a new store and add/remove shopping lists to it
  • Create a new shopping list and/remove stores to it

Handling long lists of linked entities

It’s possible that a user will add lots of stores to a shopping list (or lots of shopping lists to a store), so the list has to truncate after a to-be-determined number of items.

Already, we’re seeing some overflow after just three linked entities are present:

Currently, the logic that creates this list looks like so:

  _entityList(list) {
    var shortList = List();
    shortList.add(Text("This $listType is not attached to any $entities yet."));
    // if 'list' is empty, default to shortList which is guaranteed to have something
    return list?.map((item) => Text(item.toString(), style: TextStyle(height: 1.6)))?.toList() ?? shortList;
  }

The refactor needs to do the following:

  • display up to N (probably 4 or 5) items
  • append a Text widget showing count of how many items remain, ie: “+ 2 more”
  • still return a Text widget that says “This listType is not attached to any $entities yet” when the list is empty

Here’s what I ended up with. (There are probably more succinct ways to write this, but hopefully it’s clear what it’s doing.)

  _entityList(list) {
    const MAX_LIST_LEN = 4;

    if (list == null || list?.length == 0) {
      return [Text("This $listType is not attached to any $entities yet.")];
    } else {
      var listLen = list.length > MAX_LIST_LEN ? MAX_LIST_LEN : list.length;
      var entityList = List();
      for (var i = 0; i < listLen; i++) {
        entityList.add(Text(list[i].toString()));
      }
      if (list.length > MAX_LIST_LEN) {
        entityList.add(Text('+${list.length - MAX_LIST_LEN} more'));
      }
      return entityList;
    }
  }

The result:

Adding “store address” anywhere store names are displayed (store list, toggle list)

When I built the ToggleList and the linked entities list, I overlooked the (common) use case of the user having multiple stores with the same name. Without each store’s address on display, it’s hard to tell identically named stores apart.

However, I have a bit of a conundrum: “entities” (as they are), are just persisted to the database as a string representing the store (or shopping list’s) name. There’s no address field, nor do I really want to add one at this point.

I decided the simplest course of action would be appending the location information to the name right before it’s pushed into the database, like so: “Safeway (Kirkland)” and see how far that carries me. Is this hacky? Maybe ;) But it feels good enough for now.

In toggle_list.dart, the full list of shopping lists or stores is passed in as a Map, known as this.list:

ToggleList({Key key, @required this.parentType, @required this.parentID, @required this.parentName, @required this.list, @required this.linkedEntities});

Around line 72, each of these list items (which can be stores or shopping lists) are turned into LinkedEntity instances:

var item = LinkedEntity(widget.list[index]);

Both stores and shopping lists become instances of LinkedEntity, and they can share this “base class” because the only things used are their id and name. But now they have a third field: address. If there is an address it’s saved to the address field, but if there is no address (ie: shopping lists), it’ll just set address to be empty.

class LinkedEntity {
  String id;
  String name;
  String address;

  LinkedEntity(DocumentSnapshot document) {
    this.id = document['id'];
    this.name = document['name'];
    this.address = document['address'] ?? '';
  }
}

Then, when the list is built, the itemName is assembled out of either item.name + item.address, or just item.name if there was no address.

@override
  Widget build(BuildContext context) {

    return ListView.builder(
        shrinkWrap: true, // gives it a size
        itemCount: widget.list.length,
        itemBuilder: (BuildContext context, int index) {
          var item = LinkedEntity(widget.list[index]);
          var itemName = item.address.length > 0 ? item.name + ' (${item.address})' : item.name;
          return SwitchListTile(
            title: Text(itemName),
            value: widget.linkedEntities?.containsKey(item.id) ?? false,
            onChanged: (bool value) => toggleItem(item.id, item.name, value),
          );
        }
    );
  }

Now it’s much easier to tell same-name stores apart in the toggle list:

The same address treatment would be useful on the form page, too:

This page is trickier, because this list is taken from the saved “stores” data in the database. I considered a few different solutions, all of which felt cumbersome (on top of some already-cumbersome-feeling logic), until it dawned on me that I could just persist the “StoreName (Address)” string to the database.

It ended up being a one-line (one word, really) change in toggle_list.dart:

          return SwitchListTile(
            title: Text(itemName),
            value: widget.linkedEntities?.containsKey(item.id) ?? false,
            onChanged: (bool value) => toggleItem(item.id, itemName, value),

Now when toggleItem is called, the same “StoreName (Address)” string is passed to the database.

I’ll go with this approach for now, which seems “good enough” for the sake of this project. (I toggled each store on/off to update it to the new StoreName (Address) format.)

Two more changes…

Before moving on, I had to make two more (small) changes to the code that runs when an existing shopping list or store is updated (renamed) or when a new shopping list or store is created.

store_form.dart

void updateStore(BuildContext context) async {
    final formState = formKey.currentState;

    if (formState.validate()) {
      formKey.currentState.save();
      storeFields.date = DateTime.now().toString();

      if (widget.store != null) {
        storeFields.id = widget.store.id;
        // 1
        storeFields.shoppingLists = widget.store.shoppingLists;
        await db.updateStore(widget.store.id, storeFields);
      } else {
        // 2
        storeFields.shoppingLists = Map();
        await db.addStore(storeFields);
      }

      Navigator.of(context).pop();
    }
  }
  1. If the store already exists (it’s not null), then copy its shoppingLists into storeFields.shoppingLists and send them along to db.updateStore(...).
  2. If the store is null, then it’s a new one being created, so create a new Map() for storeFields.shoppingLists and send that along to the db. Without this, a new shopping list has “null” for its shoppingLists map and the code that adds a new shopping list ID to it fails.

I made a similar set of changes to shopping_list_form.dart.

Updating renaming shopping lists and renaming stores to also update any saved links

The last major piece of work on this feature is making it so that updating a shopping list’s name (or a store’s name) is properly propagated to all of the documents that have it stored.

As far as I can tell, having redundancies like this (such as storing a store’s name or shopping list in multiple places) is oftentimes the preferred way of storing records in Firebase.

Per my own logic, it’s fairly uncommon to change the name of a list or a store but very common to view a store or a list in multiple places. It seemed better to take on the burden of having to update multiple records with a name change once in a rare while vs. the burden of looking up the name for every store and shopping list, by ID, every time the user viewed a list of them.

This piece of work needs to achieve the following:

  • When the user changes the name of a Store, step through each existing Shopping List and look for that store.id in each Shopping List’s “stores” map. If any match is found, update the name saved for that store entry.
  • When the user changes the name of a Shopping List, step through each existing Store and look for that shoppingList.id in each Store’s “shoppingList” map. If any match is found, update the name saved for that shopping list entry.

I started my work with renaming stores first, because stores are more complicated. Stores record their name separate from their location (also called “address” in the code), but their name and location are concatenated together when saving them into a shopping list’s list of stores.

In database_manager.dart, I confirmed that the name and address are accessible on the store DTO with a couple of print statements:

  Future updateStore(String id, StoreDTO store) async {
    print(store.name);
    print(store.address);
    if (id != null && id.length > 0) {
      DocumentReference docRef = stores.document(id);
      Firestore.instance.runTransaction((transaction) async {
        await transaction.update(docRef, store.toJson());
      }).catchError((e) {
        print(e.toString());
      });
    } else {
      print("ID is null/has no length");
    }
  }

Then passed them along to a new method called updateLinkedShoppingLists:

  Future updateStore(String id, StoreDTO store) async {
    if (id != null && id.length > 0) {
      DocumentReference docRef = stores.document(id);
      Firestore.instance.runTransaction((transaction) async {
        await transaction.update(docRef, store.toJson());
      }).catchError((e) {
        print(e.toString());
      });
      updateLinkedShoppingLists(store.id, store.name + " (" + store.address + ")");
    } else {
      print("ID is null/has no length");
    }
  }

    Future updateLinkedShoppingLists(storeID, newName) async {
    // update all the shopping lists's "stores" maps to use the new store name
    await shoppingLists
        .getDocuments()
        .then((querySnapshot) => {
          querySnapshot.documents.forEach((doc) => {
            if (doc.data['stores'][storeID] != null) { // can't use ['stores.$storeID']
              doc.reference.updateData({'stores.$storeID': newName})
            }
          })
        });
  }

updateLinkedShoppingLists gets all the shopping list documents from the shoppingLists collection then iterates through them. On each one, it checks if doc.data['stores'][storeID] is not null, and if it’s not null, it updates the stores entry to have the new name.

The hardest part of this process was figuring out how to only updates the stores {"storeID": "storeName"} entry for shopping lists that actually had this store in their store map. Without this logic check, every shopping list gets the store added, whether it had it before or not, but figuring out how to limit the updateData call to just the documents that had that particular store ID in its store map was a challenge. The docs didn’t really cover this scenario, and the stores.$storeID syntax didn’t work.

In other words, I couldn’t do this:

if (doc.data['stores.$storeID'] != null) { // doesn't work

It seems the handy-dandy store.$storeID lookup is only for use in the updateData({...}) call. For checking if a Firebase document had a particular entry in a map, this was the syntax that worked:

if (doc.data['stores'][storeID] != null) { ... 

Perhaps because it doesn’t know the structure of ‘stores’ so it can’t use the dot notation. Either way, here is the “rename everywhere” feature working for Stores:

And getting it working for renaming shopping lists was as easy as writing the same thing again, but for shopping lists and their linked stores:

  Future updateShoppingList(String id, ShoppingListDTO shoppingList) async {
    if (id != null && id.length > 0) {
      DocumentReference docRef = shoppingLists.document(id);
      Firestore.instance.runTransaction((transaction) async {
        await transaction.update(docRef, shoppingList.toJson());
      }).catchError((e) {
        print(e.toString());
      });
      updateLinkedStores(shoppingList.id, shoppingList.name); // call new method
    } else {
      print("ID is null/has no length");
    }
  }

  Future updateLinkedStores(shoppingListID, newName) async {
    // update all the stores' "shopping lists" maps to use the new shopping list name
    await stores
        .getDocuments()
        .then((querySnapshot) => {
      querySnapshot.documents.forEach((doc) => {
        if (doc.data['shoppingLists'][shoppingListID] != null) {
          doc.reference.updateData({'shoppingLists.$shoppingListID': newName})
        }
      })
    });
  }

And that’s it! Here’s the feature’s pull request.

Here’s what we did:

  • Added a new view that lets the user toggle stores “on/off” for shopping lists (and shopping lists “on/off” for stores)
  • Wrote new Database Manager methods to create/delete a two-way link between stores and shopping lists whenever one is added or removed
  • Added a map to the store documents (and a map to the shopping list documents) that tracks the IDs and names of any linked entities
  • Made it so that changing the name of a store or shopping list also updates its name in any documents that have it in their linked entities map
  • Updated the store form and shopping list forms to display a list of linked entities, with special handling for cases where there are more than 4 linked entities

Go back to Part 7’s feature work list.

Building a Flutter app: “Dark mode” feature

This post is part of my “Building a Flutter App” dev journal, which begins here.

In this post: I add a settings drawer and a “Dark Mode” toggle to my app. The app saves this setting locally so that the setting persists through app restarts.

First, the final product

The user toggles between dark mode and light mode in a settings drawer.

Material UI’s built in “dark” theme

First, I wanted to see what kind of “dark theme” was already supported in Flutter/Material UI. I changed the theme property to ThemeData.dark(), like so:

return MaterialApp(
      routes: routes,
      theme: ThemeData.dark(),
      home: MainPage(title: 'Grocery Go!'),
    );

Sweet – this default palette will be fine for now.

Adding a “Settings” drawer with a “Dark Mode” toggle switch

Material UI offers a “drawer” widget for headers, and that’s what I am going to begin with. I basically took their example and dropped it into my own main.dart Scaffold, and swapped one of their ListTiles for a SwitchListTile.

main.dart

...

class _MainPageState extends State<MainPage> {

  final DatabaseManager db = DatabaseManager();

  _goToList(ShoppingList list) {
    Navigator.pushNamed(context, MainShoppingList.routeName, arguments: MainShoppingListArguments(list));
  }

  _editStore(Store store) {
    Navigator.pushNamed(context, ExistingStore.routeName, arguments: ExistingStoreArguments(store));
  }

  _editList(ShoppingList list) {
    Navigator.pushNamed(context, ExistingList.routeName, arguments: ExistingListArguments(list));
  }

  bool darkTheme = false; 

@override
  Widget build(BuildContext context) {

    const headerShoppingLists = "Shopping Lists";
    const headerStores = "Stores";

    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      drawer: Drawer(
        child: ListView(
          padding: EdgeInsets.zero,
          children: <Widget>[
            DrawerHeader(
              decoration: BoxDecoration(
                color: Colors.blue,
              ),
              child: Text(
                'Grocery Go',
                style: TextStyle(
                  color: Colors.white,
                  fontSize: 22,
                ),
              ),
            ),
            SwitchListTile(
              title: Text('Dark Mode'),
              value: darkTheme,
              onChanged: (bool value) {
                setState(() {
                  darkTheme = value;
                });
              },
            ),
            ListTile(
              leading: Icon(Icons.account_circle),
              title: Text('Account management'),
              subtitle: Text('Logged in as TILCode')
            ),
            ListTile(
              leading: Icon(Icons.settings),
              title: Text('App preferences'),
            ),
          ],
        ),
      ),
      body: LayoutBuilder(
        builder: (BuildContext context, BoxConstraints viewportConstraints) {
          return SingleChildScrollView(
            child: Column(
              mainAxisSize: MainAxisSize.min,
              mainAxisAlignment: MainAxisAlignment.start,
              children: <Widget>[
                ItemListHeader(text: headerShoppingLists),
                ItemListStream(dbStream: db.getShoppingListStream(), listType: 'shopping list', onTap: _goToList, onInfoTap: _editList),
                ItemListHeader(text: headerStores),
                ItemListStream(dbStream: db.getStoresStream(), listType: 'store', onTap: _editStore, onInfoTap: _editStore),
              ],
            ),
          );
        }),
    );

Now there’s a drawer with a “Dark Mode” toggle but it doesn’t do anything yet.

Toggling between light/dark mode

Still in main.dart, the next thing I did was make the theme conditional on the darkTheme variable. But wait, darkTheme isn’t available up here where MaterialApp is called. Hmm.

class GroceryGoApp extends StatelessWidget {

  @override
  Widget build(BuildContext context) {

    var routes = {
      ExistingList.routeName: (context) => ExistingList(),
      MainShoppingList.routeName: (context) => MainShoppingList(),
      NewShoppingList.routeName: (context) => NewShoppingList(),
      ExistingStore.routeName: (context) => ExistingStore(),
      NewStore.routeName: (context) => NewStore(),
      NewItem.routeName: (context) => NewItem(),
      ExistingItem.routeName: (context) => ExistingItem(),
    };

    return MaterialApp(
      routes: routes,
      theme: darkTheme ? ThemeData.dark() : ThemeData.light(),
      home: MainPage(title: 'Grocery Go!'),
    );
  }
}

I decided to fix by changing the GroceryGoApp class into a StatefulWidget so I could lift the darkTheme variable up into it and check its value to determine if the theme should be ThemeData.dark() or ThemeData.light().

Then, instead of passing a title string into MainPage (which wasn’t doing anything anyway), I instead pass darkTheme and toggleTheme as parameters.

class GroceryGoApp extends StatefulWidget {

  @override
  _GroceryGoAppState createState() => _GroceryGoAppState();
}

class _GroceryGoAppState extends State<GroceryGoApp> {

  bool darkTheme = false;

  void toggleTheme (bool value) {
    setState(() {
      darkTheme = value;
    });
  }

  @override
  Widget build(BuildContext context) {

    var routes = {
      ExistingList.routeName: (context) => ExistingList(),
      MainShoppingList.routeName: (context) => MainShoppingList(),
      NewShoppingList.routeName: (context) => NewShoppingList(),
      ExistingStore.routeName: (context) => ExistingStore(),
      NewStore.routeName: (context) => NewStore(),
      NewItem.routeName: (context) => NewItem(),
      ExistingItem.routeName: (context) => ExistingItem(),
    };

    return MaterialApp(
      routes: routes,
      theme: darkTheme ? ThemeData.dark() : ThemeData.light(),
      home: MainPage(darkTheme: darkTheme, toggleTheme: toggleTheme),
    );
  }
}

Next, in MainPage, I updated its parameters to take darkTheme and toggleTheme as passed in from GroceryGoApp. Here’s the entirety of the MainPage stateful widget and its state so you can see how the darkTheme and toggleTheme parameters come into MainPage and get passed to _MainPageState and used by the SwitchListTile.

class MainPage extends StatefulWidget {
  final darkTheme;
  final toggleTheme;

  MainPage({Key key, this.darkTheme, this.toggleTheme}) : super(key: key);

  @override
  _MainPageState createState() => _MainPageState(darkTheme: darkTheme, toggleTheme: toggleTheme);
}

class _MainPageState extends State<MainPage> {

  final darkTheme;
  final toggleTheme;

  _MainPageState({this.darkTheme, this.toggleTheme});

  final DatabaseManager db = DatabaseManager();

  _goToList(ShoppingList list) {
    Navigator.pushNamed(context, MainShoppingList.routeName, arguments: MainShoppingListArguments(list));
  }

  _editStore(Store store) {
    Navigator.pushNamed(context, ExistingStore.routeName, arguments: ExistingStoreArguments(store));
  }

  _editList(ShoppingList list) {
    Navigator.pushNamed(context, ExistingList.routeName, arguments: ExistingListArguments(list));
  }

  @override
  Widget build(BuildContext context) {

    const headerShoppingLists = "Shopping Lists";
    const headerStores = "Stores";

    return Scaffold(
      appBar: AppBar(
        title: Text('Grocery Go'),
      ),
      drawer: Drawer(
        child: ListView(
          padding: EdgeInsets.zero,
          children: <Widget>[
            DrawerHeader(
              decoration: BoxDecoration(
                color: Colors.blue,
              ),
              child: Text(
                'Grocery Go',
                style: TextStyle(
                  color: Colors.white,
                  fontSize: 22,
                ),
              ),
            ),
            SwitchListTile(
              title: Text('Dark Mode'),
              value: darkTheme,
              onChanged: toggleTheme,
            ),
            ListTile(
              leading: Icon(Icons.account_circle),
              title: Text('Account management'),
              subtitle: Text('Logged in as TILCode')
            ),
            ListTile(
              leading: Icon(Icons.settings),
              title: Text('App preferences'),
            ),
          ],
        ),
      ),
      body: LayoutBuilder(
        builder: (BuildContext context, BoxConstraints viewportConstraints) {
          return SingleChildScrollView(
            child: Column(
              mainAxisSize: MainAxisSize.min,
              mainAxisAlignment: MainAxisAlignment.start,
              children: <Widget>[
                ItemListHeader(text: headerShoppingLists),
                ItemListStream(dbStream: db.getShoppingListStream(), listType: 'shopping list', onTap: _goToList, onInfoTap: _editList),
                ItemListHeader(text: headerStores),
                ItemListStream(dbStream: db.getStoresStream(), listType: 'store', onTap: _editStore, onInfoTap: _editStore),
              ],
            ),
          );
        }),
    );
  }
}

Success! Well, sort of – the little toggle switch isn’t “toggling”. What’s up with that?

Fixing the SwitchListTile switch not toggling

As it turned out, this seemingly-small thing ended up occupying quite a bit of my time. The fix was to actually stop passing darkTheme and toggleTheme into _MainPageState the way I had been.

Instead, I just run an empty constructor and then, when I need to refer to darkTheme and toggleTheme, I address them as widget.darkTheme and widget.toggleTheme. (More on why after the code sample.)

class MainPage extends StatefulWidget {
  final darkTheme;
  final toggleTheme;

  MainPage({Key key, this.darkTheme, this.toggleTheme}) : super(key: key);

  @override
  _MainPageState createState() => _MainPageState();
}

class _MainPageState extends State<MainPage> {

  _MainPageState();

  final DatabaseManager db = DatabaseManager();

  _goToList(ShoppingList list) {
    Navigator.pushNamed(context, MainShoppingList.routeName, arguments: MainShoppingListArguments(list));
  }

  _editStore(Store store) {
    Navigator.pushNamed(context, ExistingStore.routeName, arguments: ExistingStoreArguments(store));
  }

  _editList(ShoppingList list) {
    Navigator.pushNamed(context, ExistingList.routeName, arguments: ExistingListArguments(list));
  }

  @override
  Widget build(BuildContext context) {

    const headerShoppingLists = "Shopping Lists";
    const headerStores = "Stores";

    return Scaffold(
      appBar: AppBar(
        title: Text('Grocery Go'),
      ),
      drawer: Drawer(
        child: ListView(
          padding: EdgeInsets.zero,
          children: <Widget>[
            DrawerHeader(
              decoration: BoxDecoration(
                color: Colors.blue,
              ),
              child: Text(
                'Grocery Go',
                style: TextStyle(
                  color: Colors.white,
                  fontSize: 22,
                ),
              ),
            ),
            SwitchListTile(
              title: Text('Dark Mode'),
              value: widget.darkTheme,
              onChanged: widget.toggleTheme,
            ),
            ListTile(
              leading: Icon(Icons.account_circle),
              title: Text('Account management'),
              subtitle: Text('Logged in as TILCode')
            ),
            ListTile(
              leading: Icon(Icons.settings),
              title: Text('App preferences'),
            ),
          ],
        ),
      ),
      body: LayoutBuilder(
        builder: (BuildContext context, BoxConstraints viewportConstraints) {
          return SingleChildScrollView(
            child: Column(
              mainAxisSize: MainAxisSize.min,
              mainAxisAlignment: MainAxisAlignment.start,
              children: <Widget>[
                ItemListHeader(text: headerShoppingLists),
                ItemListStream(dbStream: db.getShoppingListStream(), listType: 'shopping list', onTap: _goToList, onInfoTap: _editList),
                ItemListHeader(text: headerStores),
                ItemListStream(dbStream: db.getStoresStream(), listType: 'store', onTap: _editStore, onInfoTap: _editStore),
              ],
            ),
          );
        }),
    );
  }
}

This kinda blew my mind – I thought you had to explicitly pass parameters into the State object, but it turns out that’s not the case. In fact, you’re not supposed to pass them in via the constructor, and you should access them using widget.fieldName like I did here.

(Furthermore, any parameters that are passed to the State object through the constructor will never get updated. Lesson learned! I may have to update some of my other State objects.)

Persisting the user’s dark theme/light theme choice through app reload

Currently, the app defaults to “light mode” every time you reload the app. I’d like to persist the user’s preference, but I want to store it locally (on the device) instead of pushing it to the database.

Shared Preferences is one common solution – it’s a Flutter package that makes it easy to store key/value pairs locally on the device.

I followed this guide and made the following changes to my project’s code:

pubspec.yaml

...

dependencies:
  intl: ^0.16.1
  timeago: ^2.0.26
  cloud_firestore: ^0.13.6
  firebase_storage: ^3.1.6
  shared_preferences: ^0.5.7+3
  ...

(Android Studio prompted me to update after I added this line, but you can also do the update manually via the Terminal with flutter pub get).

main.dart

import 'package:shared_preferences/shared_preferences.dart';

/* GroceryGoApp now expects a 'preferences' parameter 
   I pass it an instance of SharedPreferences. */

void main() async {
  runApp(GroceryGoApp(preferences: await SharedPreferences.getInstance()));
}

// Here's GroceryGoApp receiving its instance of SharedPreferences in its constructor

class GroceryGoApp extends StatefulWidget {
  final SharedPreferences preferences;
  GroceryGoApp({Key key, @required this.preferences}) : super(key: key);

  @override
  _GroceryGoAppState createState() => _GroceryGoAppState();
}

/* 
Hey look, a chance to use my new "widget." trick - 
I don't explicitly pass preferences into State, I get them via widget.preferences...
I also replaced 'darkTheme' with a const because I like to YELL_AT_MY_CODE
(really, I just wanted to define the key name in one place in case I end up changing it, because calling it 'darktheme' in some places and 'dark mode' in others is already starting to bother me) */

class _GroceryGoAppState extends State<GroceryGoApp> {
  static const DARK_THEME_KEY = 'darkTheme';
  bool get darkTheme => widget.preferences.getBool(DARK_THEME_KEY) ?? false;

  void toggleTheme(bool value) {
    setState(() {
      widget.preferences.setBool(DARK_THEME_KEY, !darkTheme);
    });
  }

  @override
  Widget build(BuildContext context) {

The first rebuild after installing SharedPreferences was pretty slow, but when it was done – ta-dah!

Oh, no, what’s this? A bunch of error output:

Launching lib/main.dart on iPhone SE (2nd generation) in debug mode...
Running Xcode build...
Xcode build done.                                           22.9s
	path: satisfied (Path is satisfied), interface: en0
Configuring the default Firebase app...
Configured the default Firebase app __FIRAPP_DEFAULT.
	path: satisfied (Path is satisfied), interface: en0
	path: satisfied (Path is satisfied), interface: en0
[VERBOSE-2:ui_dart_state.cc(157)] Unhandled Exception: ServicesBinding.defaultBinaryMessenger was accessed before the binding was initialized.
If you're running an application and need to access the binary messenger before `runApp()` has been called (for example, during plugin initialization), then you need to explicitly call the `WidgetsFlutterBinding.ensureInitialized()` first.
If you're running a test, you can call the `TestWidgetsFlutterBinding.ensureInitialized()` as the first line in your test's `main()` method to initialize the binding.
#0      defaultBinaryMessenger.<anonymous closure> (package:flutter/src/services/binary_messenger.dart:76:7)
#1      defaultBinaryMessenger (package:flutter/src/services/binary_messenger.dart:89:4)
#2      MethodChannel.binaryMessenger (package:flutter/src/services/platform_channel.dart:140:62)
#3      MethodChannel.invokeMethod (package:flutter/src/services/platform_channel.dart:314:35)
#4      MethodChannel.invokeMapMethod (package:flutter/src/services/platfo<…>
Debug service listening on ws://127.0.0.1:64936/RkSpLN3ICpY=/ws
Syncing files to device iPhone SE (2nd generation)...

The app itself is a plain white screen. Hmm.

[Did a bunch of Googling and reading – this Stack Overflow post was the most helpful.]

This seems to happen because the app is waiting on main to do something (main is now async, so that makes sense), and the fix is to add WidgetsFlutterBinding.ensureInitialized() to the first line of main.

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  runApp(GroceryGoApp(preferences: await SharedPreferences.getInstance()));
}

With SharedPreferences in place, I can now toggle the app to “dark mode”, close the simulator, re-build, and the app is still in “dark mode” when I reopen the app. Yay!

View this feature’s pull request.

Now the user can cross items off and add them back to the active list by tapping on them. Return to the feature work list.

Building a Flutter app, part 7: in which I blast through a ton of feature work

The setup and database hookups are done, so now it’s time to just pump out features.

Each feature has its own article:

  1. Crossing off shopping list items
  2. Dark UI mode
  3. Linking stores and lists
  4. Reordering items on a per-store basis
  5. Mark an item as exclusive to a store(s) [Coming soon]
  6. Substitutes list – [Coming soon]
  7. Add a picture to an item – [Coming soon]
  8. Enhanced “store location” – [Coming soon]
  9. Deleting items, shopping lists, and stores – [Coming soon]

Coming soon: Part 8, user accounts.

Building a Flutter app: “Cross off” feature

This post is part of my “Building a Flutter App” dev journal, which begins here.

In this post: I built the “cross off” feature to my “Grocery Go” shopping list management app.

First, the final product

Watch as items move between the top list (the “active” list) and the bottom list (the “crossed off” list).

Feature list

Here are all the things that should happen when this feature is considered done:

  • When the user taps the item, it is visibly removed from the shopping list and added to the “Crossed off” list below the shopping list
  • The parent shopping list itemCount is decreased by one
  • A “crossed off” item appears at the top of the crossed off items list
  • The time/date of this action is saved to the item so we know when it was crossed off [dateLastMoved]
  • If the user taps a “crossed off” item, it returns to the shopping list and the itemCount is increased by one
  • The time/date of this action is saved to the item so we know when it was added to the list [dateLastMoved]
  • Shopping Lists maintain their own lists of Crossed Off items, so that the user doesn’t see a crossed off item from their “home improvement” list in their “groceries” list – add CrossedOff subcollection to Shopping List

Adding the isCrossedOff property to Item

I thought it might be simple to just have each item track whether it is “active” or “crossed off”. isCrossedOff will be a boolean value on Item instances, so it had to be added to any code that models Item’s data.

That includes item_dto.dart:

class ItemDTO {

  String id;
  String name;
  String date;
  int quantity;
  bool subsOk;
  List substitutions;
  String addedBy;
  String lastUpdated;
  bool private;
  bool urgent;
  bool isCrossedOff;

  String toString() {
    return 'id: $id, name: $name, date: $date, '
        'quantity: $quantity, subsOk: $subsOk, substitutions: $substitutions, '
        'addedBy: $addedBy, lastUpdated: $lastUpdated, private: $private, urgent: $urgent, isCrossedOff: $isCrossedOff';
  }

  Map<String, dynamic> toJson() => <String, dynamic> {
    'id': this.id,
    'name': this.name,
    'date': this.date,
    'quantity': this.quantity,
    'subsOk': this.subsOk,
    'substitutions': this.substitutions,
    'addedBy': this.addedBy,
    'lastUpdated': this.lastUpdated,
    'private': this.private,
    'urgent':this.urgent,
    'isCrossedOff':this.isCrossedOff,
  };
}

models/item.dart:

import 'package:cloud_firestore/cloud_firestore.dart';

class Item {
  String id;
  String name;
  String date;
  int quantity;
  bool subsOk;
  List substitutions;
  String addedBy;
  String lastUpdated;
  bool private;
  bool urgent;
  bool isCrossedOff;

  Item(DocumentSnapshot document) {
    this.id = document['id'];
    this.name = document['name'];
    this.date = document['date'];
    this.quantity = document['quantity'];
    this.subsOk = document['subsOk'];
    this.substitutions = document['substitutions'];
    this.addedBy = document['addedBy'];
    this.lastUpdated = document['lastUpdated'];
    this.private = document['private'];
    this.urgent = document['urgent'];
    this.isCrossedOff = document['isCrossedOff'];
  }
}

new_item_form.dart

    if (formState.validate()) {
      formKey.currentState.save();

      itemFields.date = DateTime.now().toString();
      itemFields.lastUpdated = DateTime.now().toString();
      itemFields.addedBy = "TILCode";
      itemFields.subsOk = true;
      itemFields.substitutions = new List<String>();
      itemFields.private = false;
      itemFields.quantity = 1;
      itemFields.urgent = false;
      itemFields.isCrossedOff = false;

      var docRef = await db.createItem(args.parentListID, itemFields);

and edit_item_form.dart:

  @override
  void initState() {
    print(args.item.date.toString());
    itemFields.id = args.item.id;
    itemFields.name = args.item.name;
    itemFields.addedBy = args.item.addedBy;
    itemFields.date = args.item.date;
    itemFields.quantity = args.item.quantity;
    itemFields.subsOk = args.item.subsOk;
    itemFields.private = args.item.private;
    itemFields.urgent = args.item.urgent;
    itemFields.isCrossedOff = args.item.isCrossedOff;
    return super.initState();

To avoid having to delete my existing items and re-create them through the form just now, I also went into their Firebase documents and added isCrossedOff to my three test items:

Filtering the item stream by isCrossedOff’s value

Next up: adding a second “get item stream” method to database_manager.dart.

Originally, getItemsStream() returned all the items. Now, I want it to only return items where isCrossedOff is set to false, like so:

Stream<QuerySnapshot> getItemsStream(shoppingListID) {
    return shoppingLists.document(shoppingListID).collection('items').where('isCrossedOff', isEqualTo: false).snapshots();
}

A second stream method can return items where crossedOff is true.

Stream<QuerySnapshot> getCrossedOffStream(shoppingListID) {
    return shoppingLists.document(shoppingListID).collection('items').where('isCrossedOff', isEqualTo: true).snapshots();
  }

Back in main_shopping_list.dart I added another use of my ItemListStream widget and pass it db.getCrossedOffStream for the given shopping list ID:

return SingleChildScrollView(
  child: Column(
    mainAxisSize: MainAxisSize.min,
    mainAxisAlignment: MainAxisAlignment.start,
    children: <Widget>[
      ItemListHeader(text: args.list.id), 
      ItemListStream(dbStream: db.getItemsStream(args.list.id), listType: 'item', onTap: _crossOff, onInfoTap: _editItem, parentList: args.list),
      ItemListHeader(text: "Crossed off"),
      ItemListStream(dbStream: db.getCrossedOffStream(args.list.id), listType: 'item', onTap: _moveBack, onInfoTap: _editItem, parentList: args.list),
    ],

Now the items are being filtered by their isCrossedOff value:

Once I saw it working and was satisfied that I could filter the items this way, I decided to do a quick refactor to turn getItemsStream() in database_manager.dart into one method that takes a second parameter indicating whether I want the active items or the crossed off items.

  Stream<QuerySnapshot> getItemsStream(shoppingListID, isCrossedOff) {
    return shoppingLists.document(shoppingListID).collection('items').where('isCrossedOff', isEqualTo: isCrossedOff).snapshots();
  }

Which looks like this when I call getItemsStream():

 ItemListStream(dbStream: db.getItemsStream(args.list.id, false), listType: ...

Making crossed off items look “crossed off” by drawing a line through them

This step was easy – in main_shopping_list.dart, I changed the existing listType param from ‘item’ to ‘crossedOff’.

ItemListStream(dbStream: db.getItemsStream(args.list.id, true), listType: 'crossedOff', ... 

In item_list.dart, I added an additional check for the ‘crossedOff’ list type in the itemBuilder so that an Item instance it still created for it.

...

var listItem;

if (listType == 'shopping list') {
  listItem = ShoppingList(list[index]);
} else if (listType == 'store') {
  listItem = Store(list[index]);
} else if (listType == 'item' || listType == 'crossedOff') {
  listItem = Item(list[index]);
} else {
  print("Unhandled list item type");
}

list_item.dart was already set up to apply a lineThrough style if the listType is ‘crossedOff’.

return ListTile(
      title: Text(buildTitleString(), style: (listType == 'crossedOff' ? TextStyle(decoration: TextDecoration.lineThrough) : TextStyle(decoration: TextDecoration.none))),
...

Crossed off text result:

Moving items between the “active” list and the “crossed off” list

In main_shopping_list.dart, there are two methods that get passed all the way down to the Item instances:

  _crossOff(Item item) {
    print("Remove this id from this list: " + item.id);
  }

  _addToList(Item item) {
    print("Moving this item back to the list: " + item.id);
  }

They are fed into the ItemListStream as onTap: methods. Items that are ‘active’ get the _crossOff method:

ItemListStream(dbStream: db.getItemsStream(args.list.id, false), listType: 'item', onTap: _crossOff, onInfoTap: _editItem, parentList: args.list),

Items that are on the ‘crossedOff’ list get the _addToList method:

ItemListStream(dbStream: db.getItemsStream(args.list.id, true), listType: 'crossedOff', onTap: _addToList, onInfoTap: _editItem, parentList: args.list),

In my first attempt at this, I ended up with two methods that almost looked identical:

    _crossOff(Item item) async {
      await db.updateItemCrossedOffStatus(
          args.list.id,
          item.id,
          {
            'isCrossedOff': true,
            'lastUpdated': DateTime.now().toString()
          }
        );
    }

    _addToList(Item item) async {
      await db.updateItemCrossedOffStatus(
          args.list.id,
          item.id,
          {
            'isCrossedOff': false,
            'lastUpdated': DateTime.now().toString()
          }
      );
    }

They both call the same DatabaseManager method:

  Future updateItemCrossedOffStatus(String parentListID, String itemID, data) async {
    if (parentListID != null && parentListID.length > 0) {
      // adjust the shopping list's item count accordingly
      if (data['isCrossedOff']) {
        shoppingLists.document(parentListID).updateData({'itemCount': FieldValue.increment(-1)});
      } else {
        shoppingLists.document(parentListID).updateData({'itemCount': FieldValue.increment(1)});
      }
      // update the item itself
      DocumentReference itemDocRef = shoppingLists.document(parentListID).collection('items').document(itemID);
      Firestore.instance.runTransaction((Transaction tx) async {
        await tx.update(itemDocRef, data);
      }).catchError((e) {
        print(e.toString());
      });
    } else {
      print("ID is null/has no length");
    }
  }

I refactored them into one method that just flips whatever the item’s current value for isCrossedOff currently is:

    _updateCrossedOffStatus(Item item) async {
      await db.updateItemCrossedOffStatus(
          args.list.id,
          item.id,
          {
            'isCrossedOff': !item.isCrossedOff,
            'lastUpdated': DateTime.now().toString()
          }
        );
    }

Note: I initially set out to use the “ItemDTO” for this update, but I couldn’t figure out how to use it “partially” – ie, the only parts of the item I want to update are the lastUpdated and isCrossedOff fields, but the ItemDTO requires all of the fields to be present. I could copy the entire item into it, but I wonder if there’s just some better way to do this in general…

View this feature’s pull request.

Now the user can cross items off and add them back to the active list by tapping on them. Return to the feature work list.

Building a Flutter app, part 6: adding create, read, and update (“CRUD”) operations

Welcome to Part 6 of my Flutter app dev journal. Now that the Firebase connection is working, it’s time to add some “create”, “read”, and “update” functionality.

Full disclosure: I’m saving “delete” for later. This is just the “CRU” part of “CRUD”.

Seeding the database with the first bit of data

I decided to start with “shopping lists”, so the first collection I created in Firebase was shopping_lists and I gave it one document. Now there’s something to get from the database, and a collection to add to when new ones are made.

Note: I made an id field and copied the document’s ID into it so that the ID would be easily accessible on the record’s object.

Creating a database manager class

The DatabaseManager class is a singleton that’ll get imported into any file that needs to interact with the database. In my project, it is located in lib/db/database_manager.dart

This is the absolute simplest thing I could think of – all this does is get the shopping list records as a stream. I modeled it (loosely) on the examples found in this helpful tutorial.

database_manager.dart

import 'package:cloud_firestore/cloud_firestore.dart';

class DatabaseManager {

  final CollectionReference shoppingLists = Firestore.instance.collection('shopping_lists');

  Stream<QuerySnapshot> getShoppingListStream() {
    return shoppingLists.snapshots();
  }
}

Using the database manager to get records from the Firebase database

I went back to main.dart and imported the DatabaseManager.

import './db/database_manager.dart';

Then, in the “return”, I added reference to a _shoppingLists widget:

return SingleChildScrollView(
            child: Column(
              mainAxisSize: MainAxisSize.min,
              mainAxisAlignment: MainAxisAlignment.start,
              children: <Widget>[
                ItemListHeader(text: headerShoppingLists),
                _shoppingLists(context),  <--- this is what's new

And then I created the new _shoppingLists widget, still in main.dart. It uses the StreamBuilder to get records (which may come in piecemeal from the database) and either display them as an ItemList or display an error.

  Widget _shoppingLists(BuildContext context) {
    return StreamBuilder(
        stream: db.getShoppingListStream(),
        builder: (context, snapshot) {
          if (snapshot.hasError) {
            return Text('Error: ${snapshot.error}');
          }

          if (snapshot.hasData && !snapshot.data.documents.isEmpty) {
            return ItemList(list: snapshot.data.documents, listType: 'shopping list', onItemTap: _goToList, onInfoTap: _editList);

          } else {
            return Text("Error: no shopping list data");
          }
        }
    );
  }

I had to update ItemList to create a ListItem of the correct type, like so:

return ListView.builder(
        shrinkWrap: true, // gives it a size
        primary: false, // so the shopping and store lists don't scroll independently
        itemCount: list.length + 1,
        itemBuilder: (BuildContext context, int index) {
          if (index == list.length) {
            if (listType == 'crossedOff') {
              return DeleteAll();
            } else { // store, shopping list
              return AddNew(list: list, listType: listType);
            }
          } else {
            var listItem;

            if (listType == 'shopping list') {
              listItem = ShoppingList(list[index]);
            } else if (listType == 'store') {
              //listItem = Store(list[index]);
            }

            return ListItem(item: listItem, listType: listType, count: getCount(listItem), onTap: onItemTap, onInfoTap: onInfoTap);
          }
        }
    );

And I had to update the ShoppingList model in shopping_list.dart to build itself from a document, like so:

import 'package:cloud_firestore/cloud_firestore.dart';

class ShoppingList {
  String id;
  String name;
  String listType = 'shopping list';
  List itemIDs;

  ShoppingList(DocumentSnapshot document) {
    this.id = document['id'];
    this.name = document['name'];
    this.itemIDs = document['itemIDs'];
  }
}

This is when my database permissions error became obvious, as no data was actually coming in from the db even though I was ready to display it in the Flutter app.

Troubleshooting Firebase access denied (“ConnectionState.waiting” always being true)

When I tried to get my data from the db, I got a “Missing or insufficient permissions” error.

I checked the value of snapshot.connectionState and found that it was equal to ConnectionState.waiting all the time.

  Widget _shoppingLists(BuildContext context) {
    return StreamBuilder(
        stream: db.getShoppingListStream(),
        builder: (context, snapshot) {
          if (snapshot.hasError) {
            return Text('Error: ${snapshot.error}');
          }

          if (snapshot.connectionState == ConnectionState.waiting) {
            return Text("waiting is true!");
          }
      ...

This StackOverflow post was helpful. This is where I discovered a newly created Firebase database does not allow access to anyone. It’s locked down by default.

The quick fix is to make read/write open to anyone.

By default, the rules are:

rules_version = '2';
service cloud.firestore {
  match /databases/{database}/documents {
    match /{document=**} {
      allow read, write: if false;
    }
  }
}

I changed them to:

rules_version = '2';
service cloud.firestore {
  match /databases/{database}/documents {
    match /{document=**} {
      allow read, write: if true;
    }
  }
}

This is obviously a bad idea in the long term, but I should be able to change the rules to allow registered, authorized users access to their own records (and deny everyone else) and I plan to build that soon, so for now this is acceptable.

Here’s where we’re at now:

“Hardware store” and “Groceries” are from the database, so this is forward progress even if we lost the “Stores” list in the process.

“Stores” broke because the ItemList widget expects to be fed a document snapshot, and I am going to fix that next.

This is mostly a repeat of the steps I just did to make shopping_lists. First, I added a stores collection and populated it with one store…

And then I added a way to retrieve the store record(s) in database_manager.dart:

import 'package:cloud_firestore/cloud_firestore.dart';

class DatabaseManager {

  final CollectionReference shoppingLists = Firestore.instance.collection('shopping_lists');
  final CollectionReference stores = Firestore.instance.collection('stores');
  
  Stream<QuerySnapshot> getShoppingListStream() {
    return shoppingLists.snapshots();
  }

  Stream<QuerySnapshot> getStoresStream() {
    return stores.snapshots();
  }
}

And finally, create a _stores widget in main.dart that does the same thing _shoppingLists does:

Widget _stores(BuildContext context) {
    return StreamBuilder(
        stream: db.getStoresStream(),
        builder: (context, snapshot) {
          if (snapshot.hasError) {
            return Text('Error: ${snapshot.error}');
          }

          if (snapshot.hasData && !snapshot.data.documents.isEmpty) {
            return ItemList(list: snapshot.data.documents, listType: 'store', onItemTap: _editStore, onInfoTap: _editStore);

          } else {
            return Text("Error: no store data");
          }
        }
    );

I also had to update the Store model in store.dart:

import 'package:cloud_firestore/cloud_firestore.dart';

class Store {
  String id;
  String name;
  String address;

  Store(DocumentSnapshot document) {
    this.id = document['id'];
    this.name = document['name'];
    this.address = document['address'];
  }
}

Now we have stores coming in from the database, too. Yay!

Adding new shopping lists and stores via the in-app forms

Creating the data manually in Firestore’s database dashboard is no fun, so it’s time to hook up the in-app forms to the real database.

I am going to begin by adding the “create a new shopping list” feature.

Data transfer object

For interactions with the database I like to use what’s called a “data transfer object”, it’s just a way of making sure the data sent to the db confirms to a certain structure. I created a new file, shopping_list_dto.dart and built it out as so:

class ShoppingListDTO {

  String name;
  String date;
  List itemIDs;

  String toString() {
    return 'name: $name, date: $date, storeIDs: $itemIDs';
  }

  Map<String, dynamic> toJson() => <String, dynamic> {
    'name': this.name,
    'date': this.date,
    'itemIDs': this.itemIDs,
  };
}

The “toJson” method will be useful when we need to format the data for insertion into the database.

Next, I modified new_shopping_list.dart:

import 'package:flutter/material.dart';
import 'package:grocery_go/db/database_manager.dart';
import 'package:grocery_go/db/shopping_list_dto.dart';

class NewShoppingList extends StatefulWidget {

  static const routeName = '/newShoppingList';

  NewShoppingList({Key key});

  @override
  _NewShoppingListState createState() => _NewShoppingListState();
}

class _NewShoppingListState extends State<NewShoppingList> {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text("Add new shopping list"),
      ),
      body: Center(
        child: Padding(
          padding: EdgeInsets.all(20),
          child: AddShoppingListForm(),
        ),
      ),
    );
  }
}

class AddShoppingListForm extends StatefulWidget {
  @override
  _AddShoppingListFormState createState() => _AddShoppingListFormState();
}

class _AddShoppingListFormState extends State<AddShoppingListForm> {
  final formKey = GlobalKey<FormState>();

  final DatabaseManager db = DatabaseManager();

  final newShoppingListFields = ShoppingListDTO();

  String validateStringInput(String value) {
    if (value.isEmpty) {
      return 'Please enter a name';
    } else return null;
  }

  void saveNewList(BuildContext context) {
    final formState = formKey.currentState;
    if (formState.validate()) {
      // save the form
      formKey.currentState.save();
      // this data is auto-generated when a new list is made
      newShoppingListFields.date = DateTime.now().toString();
      newShoppingListFields.itemIDs = new List<String>();
      // put this stuff in the db
      db.addShoppingList(newShoppingListFields);
      // confirm with a snack bar
      Scaffold.of(context).showSnackBar(
          SnackBar(content: Text('New list created: ' + newShoppingListFields.name))
      );
      // go back to main view
      Navigator.of(context).pop();
    }
  }

  @override
  Widget build(BuildContext context) {
    return Form(
      key: formKey,
      child: Column(
        children: [
          TextFormField(
              autofocus: true,
              decoration: InputDecoration(
                  labelText: 'Shopping list name',
                  border: OutlineInputBorder()
              ),
              validator: (value) => validateStringInput(value),
              onSaved: (value) {
                newShoppingListFields.name = value;
              }
          ),
          RaisedButton(
            onPressed: () => saveNewList(context),
            child: Text('Save'),
          ),
        ],
      ),
    );
  }
}

What’s new:

  • Changed the “form” into an actual Form widget so it can behave like a proper Flutter form
  • Instantiates an instance of ShoppingListDTO called newShoppingListFields and fills it out using form data
  • Added a form key (which is how Flutter distinguishes forms from each other)
  • Added validator property to TextFormField and created a simple validation method (all it does is check that there’s any input at all)
  • Added onSaved property to TextFormField so it knows what to do when the form is saved
  • Added saveNewList method that runs validation and, if valid, sends this form data off to the db and returns the user to the main screen
  • Changed the way the date is generated, it’s now converted to a string before it goes to the db

Getting this all working took a bit of trial and error. I experimented with Timestamp and DateTime objects before settling on pushing the date to the db as a string. I also had to try a few things before I figured out how to push the empty itemIDs array into the db in a way that would be recognized (on retrieval) as having a length.

Here it is! Now the user can create a new list and see it on the main screen.

And here it is in the database:

Just one thing is missing: the form-created shopping list needs to have its “id” field filled in after the record is created, so that’s next up.

Grabbing the new record’s ID and saving it to the record

I’d like every document (so every item, every shopping list, every store, etc.) to store its own ID in a data field. This should be useful for editing entries later on.

I wasn’t completely sure how to approach this at first – there is no ID until the document is created, so I have to created something in order to get that ID back.

Fortunately, Firebase has some documentation covering this use case. When you call the .add method, a DocumentReference is returned.

I’m already using .add, like so:

  Future<DocumentReference> addShoppingList(ShoppingListDTO shoppingList) {
    return shoppingLists.add(shoppingList.toJson());
  }

But I don’t have an update function yet, so I created that next:

  Future<void> updateShoppingList(String id, ShoppingListDTO shoppingList) {
    return shoppingLists.document(id).updateData(shoppingList.toJson());
  }

This lets me grab ID that comes back and immediately update the document to have that ID, like so (in new_shopping_list.dart):

void saveNewList(BuildContext context) async {
  final formState = formKey.currentState;
  if (formState.validate()) {
    // save the form
    formKey.currentState.save();

    // this data is auto-generated when a new list is made
    newShoppingListFields.date = DateTime.now().toString();
    newShoppingListFields.itemIDs = new List<String>();

    // put this stuff in the db and get the ID that was created
    DocumentReference docRef = await db.addShoppingList(newShoppingListFields);

    // update the record to have the ID that was generated
    newShoppingListFields.id = docRef.documentID;
    db.updateShoppingList(docRef.documentID, newShoppingListFields);

    // confirm it with a snack bar
    Scaffold.of(context).showSnackBar(
        SnackBar(content: Text('New list created: ' + newShoppingListFields.name))
    );

    // go back to main view
    Navigator.of(context).pop();
  }
}

Now the document’s ID is duplicated into a field on that document:

Refactoring it a bit…

This works, but I don’t like that my DatabaseManager is basically foisting this work onto whatever code is calling it. We’re never going to create a document and then not immediately turn around and slap the ID into it, so I wanted to see if I could put encapsulate this work within database_manager.dart

Initially, I ran into trouble trying to create a DocumentReference – a Future<DocumentReference> is not a DocumentReference, it seems.

This might be a job for async/await, which I’ve used in JavaScript/TypeScript but have not yet attempted in Flutter/Dart, so here we go – now it’s async/awaited and returning that DocumentReference.

  Future<DocumentReference> addShoppingList(ShoppingListDTO shoppingList) async {
    DocumentReference docRef = await shoppingLists.add(shoppingList.toJson());
    shoppingLists.document(docRef.documentID).updateData({'id':docRef.documentID});
    return docRef;
  }

And then over here in new_shopping_list.dart, I removed the “update the ID” code:

void saveNewList(BuildContext context) async {
  final formState = formKey.currentState;
  if (formState.validate()) {
    // save the form
    formKey.currentState.save();

    // this data is auto-generated when a new list is made
    newShoppingListFields.date = DateTime.now().toString();
    newShoppingListFields.itemIDs = new List<String>();

    // put this stuff in the db
    var docRef = await db.addShoppingList(newShoppingListFields);
    print("Created record: " + docRef.documentID);

    // confirm it with a snack bar
    Scaffold.of(context).showSnackBar(
        SnackBar(content: Text('New list created: ' + newShoppingListFields.name))
    );

    // go back to main view
    Navigator.of(context).pop();
  }
}

(I later cleaned it up by removing var docRef = and the print statement, I just wanted those to confirm that everything was working the way I expected.)

And there we have it – now the work of updating the ID is done by the DatabaseManager, which I think is just a better design for this particular use case.

All of the code pertaining to hooking up to Firebase and getting creation and retrieval working can be found in this pull request. (Hey, we’re halfway to a CRUD app!)

Sorting the shopping lists by name (alphabetically) as they come in from Firebase

Before we move on and Firebase-ify the rest of the app, I want to fix the way shopping lists appear in a seemingly random (or at least unpredictable) order.

In the long run, it’d be nice if the user could re-order these lists, but for now, I think I’ll sort them alphabetically.

Firebase has .orderBy, but it took me a bit of trial and error to realize I had to apply it to the collectionReference, not the part where we get that reference in the first place (so not the Firestore.instance.collection("collectionName") part.

Like this:

class DatabaseManager {

  final CollectionReference shoppingLists = Firestore.instance.collection('shopping_lists');
  final CollectionReference stores = Firestore.instance.collection('stores');

  Stream<QuerySnapshot> getShoppingListStream() {
    return shoppingLists.orderBy("name").snapshots();
  }

  Stream<QuerySnapshot> getStoresStream() {
    return stores.orderBy("name").snapshots();
  }

  ...

Which results in the shopping lists being sorted by name:

Minor thing, but it was bothering me the way new ones didn’t seem to have any rhyme or reason to where they ended up in the list.

Hooking up the rest of the app to the database

Making the rest of the app work with Firebase was a decent amount of work, and most of it was a re-hash of what was already done above, but I still broke the major steps into individual pull requests for anyone interested in seeing them.

Subcollections

Working with Items (items being things like “eggs”, “bread”, etc.) made it apparent that they should be stored as a subcollection of a Shopping List, rather than have their IDs saved in an array on shopping list and retrieved separately.

Firebase seems to prefer you just stick the child document(s) inside their parent documents, rather than maintain a list of child document IDs the way I’m used to doing with MySQL databases.

In Firebase, the shopping list’s ‘items’ subcollection looks like this:

To create an item and add it to the subcollection, database_manager.dart now has the following method:

  Future<DocumentReference> createItem(String parentListID, ItemDTO item) async {
    // 1
    shoppingLists.document(parentListID).updateData({'itemCount': FieldValue.increment(1)});
    // 2
    DocumentReference itemDocRef = await shoppingLists.document(parentListID).collection('items').add(item.toJson());
    // 3
    itemDocRef.updateData({'id':itemDocRef.documentID});
    return itemDocRef;
  }
  1. Gets the parent shopping list document by its ID and updates “itemCount” (since we don’t have the itemIDs array to get the length of anymore)
  2. Gets the parent shopping list document by its ID, gets the collection of ‘items’ within, and adds the new Item (as JSON) to that subcollection of items
  3. Immediately gives that new Item its own ID as a field called ‘id’

The app now uses real data from the Firebase db for its shopping lists, items, and stores!

At this point the app has the most basic “create”, “read”, and “update” support for shopping lists, stores, and items, but the app needs a whole bunch of feature love to feel more polished.

Join me in Part 7 [Coming soon] as I add a bunch of new features.