CS362 – Today I Learned… in Code

This post is part of an ongoing series recapping my experience in Oregon State University’s eCampus (online) post-baccalaureate Computer Science degree program. You can learn more about the program here.

Six-word summary: The nadir of OSU’s CS program

It’s not a great class. It’s not even a good one, unfortunately.

I have no idea how you dedicate 10 weeks to the topic of testing and spend about one sentence on the topic of Test Driven Development and absolutely zero time exposing students to any of the zillions of existing testing tools.

If you disliked CS290 for its obtuseness and CS361 for its general irrelevancy, I have bad news for you: CS362 is like they took the worst of those classes and combined them into one tedious, time-consuming course that doesn’t teach much that’s relevant to real world programming.

All this class does is have students write the same kinds of tests on the same (broken) code over and over and over until the end of the quarter. If you walk away from this class hating testing or unconvinced of its importance, please know that this class isn’t representative of what it’s like to write and maintain tests in a professional capacity.

CS362 Class Structure

10 weeks
5 code-writing assignments
Unproctored midterm and unproctored final
The occasional quiz, with no real consistency in length or content (most are 10 minutes of multiple choice but – surprise – one quiz is actually a standalone coding project, and you won’t know what you’re in for until you open the quiz!)
The way the class has you use Git/GitHub suggests they don’t understand how versioning works
Due dates and time allocations don’t follow much of a pattern
A large group project at the end where three students total form up a group and write more of the same tests
Very little that resembles “real world” testing practices

Repetitive assignments

Essentially, students spend the quarter doing five variations on the same project.

Assignment 1: Fork the class repo from GitHub and study the provided, bug-ridden Dominion code (this isn’t really an assignment in the way the ones that follow are).
Assignment 2: Refactor a small portion of the Dominion card code into separate methods (this part is basically copy/paste) and introduce five bugs of your choosing and document every change you make in a .pdf (the Dominion codebase was already packed with bugs, there was no reason to further break it)
Assignment 3: Write unit tests for your refactored Dominion card methods. Document your tests in a separate .pdf file. This assignment was an absolute beast in terms of scope and time required, and misconstrues unit tests as integration tests. Unit tests should test a small portion of the code’s behavior. These tests run the entire method and then verify that a variety of things either changed or did not change.
Assignment 4: Write “random tests” for your refactored Dominion card methods. This assignment was another beast and it basically repeats Assignment 3 except now you’re randomizing the inputs and running many more iterations.
Assignment 5: Test a classmate’s code with your tests. This assignment was unnecessary and painful. It essentially asks you to repeat Assignments 3 and 4 on a slightly different codebase (your teammate’s), which requires hours of tedious line-by-line updates to hundreds of lines of tests. The point of this assignment was to demonstrate that a change to the underlying codebase might require a change to the tests, but they could have made that point with far less busywork.
Group Assignment Part A: The instructor hand-picked 11 bugs from the “Find Dominion Bugs” Piazza thread that everyone wrote hundreds of bugs into earlier in the quarter. Students form up into groups of 3 and produce a .pdf documenting the ways they plan to capture those 11 bugs in a collection of “unit” tests.
Group Assignment Part B: Actually fix the bugs identified in Part A. Commit the functioning test code to the team’s shared GitHub repo. Turn in 40+ pages of documentation on every single bug, fix, and decision made.

You don’t get to fix any bugs until the very end of the class, and you only get to fix a few. It’s very unsatisfying to write endless tests without fixing anything. I fixed things to see if my tests were any good, but I had to put the code back to its broken state before turn-in. There’s no sense of progression.

Also, the numbering and due dates don’t follow a strict pattern so double check those due dates.

The class’s git practices are a mess

This class does more with Git/GitHub than any other OSU class, but it doesn’t teach it nor does it do much to help students who are struggling with it.

If you are new to git, do yourself a huge favor and get used to it before you get to this class. Use it for classwork and personal projects. GitHub lets you make private repos now. This class won’t teach git, but it will expect you to be proficient enough to fork a repo, make branches, and push/pull to them, and you’ll have to do it in a group setting by the second half of the course, so start getting familiar with it now. Don’t burden your group mates with teaching it to you (or fixing your mess) at the 11th hour.

There was one particular “bad practice” I think this class could improve. Throughout the course, the assignments instruct students to make multiple copies of the “dominion” folder, which is weird and unnecessary in a versioning system.

By the end of the quarter, my repo looked like this:

dominion/ <– pristine untouched copy of dominion game code
projects/
- robertwen/ <– the teacher’s folder, untouched in my repo
  - dominion/ <– another untouched copy
- granaman/ <– me
  - dominion/ <– where my work was done
  - FinalProject-BugFree/
    - dominion/ <– group tests and bug fixes
  - FinalProject-Bugs/
    - dominion <– group tests, no bug fixes

There is no reason for the dominion folder to exist in 5 different places in the project repo. Two of those versions are completely untouched throughout the class.

Branches and pull requests could have handled versioning for each assignment, and if anyone needed to refer back to the original code it was always available in the teacher’s repo (the one we all forked our copies from) or by checking out an earlier version of the repo.

I think this poor organization sets a bad example for students, many of whom are still new to git and were confused as to why the same directory was copied into multiple locations.

No feedback on your work

Unfortunately, it’s rare to get feedback on your code in this program in general. CS362 is not unique in that the TAs don’t give suggestions for improving your coding.

But this lack of feedback is exceptionally disappointing in a class that’s has you write literally hundreds of lines of tests of your own design only to hear NOTHING back.

Students are given a lot of freedom in this class, but they’ll never know if they were on the right track or not. I wrote a ton of code in this class and sent it into a void, never to be heard from again. I got an A on most of it, but maybe that’s because the accompanying document was 30 pages long.

The one time I didn’t get an A on an assignment was because the TA marked me down 15% for exposing a crash bug that was in the dominion code to begin with. At that point I didn’t know what was real anymore.

If we aren’t writing tests to expose bugs then why ARE we writing them?

I don’t know.

But I can tell you what I didn’t do in the assignments that built on that assignment: I didn’t expose that crash bug again.

I let that bug sit there, undocumented and unacknowledged, in all future assignments because to test it would be to turn in a “crash bug” that would deduct 15% off my grade each time.

I’ve worked on software for over a decade and the idea of ignoring a crash bug and knowingly failing to cover it with a test is ridiculous. Like, fireable offense ridiculous. Nobody who takes pride in their work or values transparency does this. I died a little bit inside.

But that’s the kind of class CS362 is. :(

Testing is misrepresented

If you’re hoping for help with coding style, theory, practical approaches, “what to do vs. what not to do”, you won’t find it here. What this class calls “unit testing” is more like integration testing, and little effort is made to explain the difference.

There is no TA feedback on work produced in this class. Again, that’s not a surprise, but the work in this class seems designed to invite feedback, especially since the assignments are just variations on each other. Surely the TA looked at my code and had at least one idea for improving it? I don’t think I’ve ever been in a code review of a substantial amount of code where literally nothing was suggested.

Every assignment is accompanied by pages (often 20 or more) of written documentation. There is no need to write this much documentation on code that is, for the most part, self-evident in what it is doing. I mean, there’s a string baked into every test announcing its purpose – how much more help does a TA need to figure out what the code is doing? Plus, nobody writes (or reads) this kind of stuff in the real world. They expect your code to be self-documenting and, if a process does have to be documented, it’s expected to be concise, not bloated for length’s sake.

This class doesn’t mention any existing industry-standard testing libraries and tools. I thought they could at least mention test runners, configuring tests to run automatically as part of a deployment pipeline, mocking data and services for tests, and in a 10 week class there’s time to really dig into at least one or two assert libraries.

This class has students write tests in the style of integration tests but calls them unit tests. This just made me cranky – a unit test tests a very small portion of the code. Like, a unit test tests a method that only does one job. The methods we tested were larger than that, like 25-100 lines long, and made multiple changes to the game state via their own code and calls to other methods. That’s more of an integration test at that point.

CS362 Group work

The last two assignments require students to work in self-selected groups of 3 to write unit tests and fix bugs (identified and assigned by the instructor) in the Dominion code.

There is a special place in software developer hell for the group member who dumped their unreviewed code into our repo hours before the project was due and then went out to dinner. This isn’t unique to CS362 but it’s ridiculous the way every other group in this CS program contains a student who just doesn’t give a shit. This student promises everything and produces nothing, turns in work hours before the deadline, and acts like it’s no big deal that their work is garbage.

In CS362, they split the last project into two parts (A and B), made B depend on the work done in A, but gave them both the exact same due date. So, of course, one of our teammates saved all of their work on A and B for the last day and turned in a bunch of grade-dropping junk at the 11th hour.

The very least they could do in CS362 is make it so Part A is due before Part B so the functioning members of the team can identify the non-performing team member earlier than the day it’s all due.

What’s missing from CS362

Test-driven development. This class could’ve given students a simple project and asked them to write and turn in the tests before writing the project code. If they still want students to write reams of documentation, an assessment of how well those tests “held up” once the program was written might be valuable.

Expose students to at least one existing test/assert library. There are so many to pick from. Pick anything, it doesn’t matter, the point is to show students that test libraries exist and get them accustomed to writing in their unique syntax. My first assert library was Chai: https://www.chaijs.com/api/assert/ and I named (and structured) my C test methods for CS362 similarly.

Teach more testing know-how. The class could cover mocks and stubs, unit and integration tests, and how to write a bug. The class tries to cover “how to write a bug” but the only thing you’re really graded on is how well you followed the bug reporting template they provided, so don’t leave off irrelevant fields (such as “browser version” for a program that runs as a standalone executable) or you’ll lose points.

Show students how testing can be part of an automated build/deploy pipeline. Testing in this class kinda comes off as a thing you do locally on your machine when/if you modify the codebase and then forget about. It kinda blew my mind when I started working to see that tests were actually the build’s gatekeeper – that is, the deploy didn’t happen if the automated tests didn’t pass, and the tests were run in their own environment and occasionally subject to failures that didn’t happen on our dev machines. This topic could be its own post, but it might be a nice thing to expose students to.

Pass on some testing “wisdom”. One of my favorites is “if you haven’t seen it fail, then you haven’t really seen it pass“. That is, if your test passes right out the gate, don’t move on right away – perhaps it always passes?

But don’t just take my word for it…

If you’re used to checking the Course Explorer to scope out a class ahead of time, be warned that most of the reviews for CS362 are from a previous version of the class. They’re not accurate anymore. In the meantime, here is some real (anonymized) commentary from the class Slack chat.