2019 Learning Recap

As we say goodbye to 2019, I wanted to make a little record of some of the cool things I worked on and learned this year.

I made a web app in React!

I’m very far from a React expert, but I really wanted to at least get my feet wet (kiddie pool, not ready for the ocean yet) in it because React is so popular.

Looking back on this project, I wish I’d documented my trial and error, but it was a lot of trial and error – I’m nowhere near ready to write a tutorial on it.

I used React to build a webapp that checks a blog post you give it for defunct Amazon links:

It scrapes the post you give it for Amazon links and feeds them through Amazon’s advertising API to see if a product comes back or not. If a product does not come back, the link is marked as suspicious in the report it generates. (Some products exist on Amazon.com but are not in their API for some reason – so it does generate some false negatives. Hey, it still beats manually clicking each link.)

It’s live – I deployed it to a free Heroku server (hence the slow startup).

I got my start in web development setting up and running affiliate marketing blogs, and I always thought it’d be neat to have a tool that can check for dead links instead of periodically clicking them all myself.

You have to be an established affiliate with Amazon with at least a small amount of monthly sales to get access to their product API, so I made a video demonstrating how it works for those who don’t have credentials and/or just want to see it in action.

The biggest thing I got out of this project was the experience of revising my initial approach several times. My initial idea was to scrape the blog post for Amazon links, then basically run automated tests on each link to see if an Amazon page loaded and scraping it for data to determine if the product is actually there. Whoops – Amazon does not like being scraped. They direct you towards their API instead, which was a whole new project-within-a-project.

There were many times during this tool’s development where I hit a wall and thought I’d abandon it, only to have an idea later that day for working around whatever limitation I was up against.

I made my first WordPress plugin!

I started using WordPress way back in 2013. I have lots of experience using plugins, but I’ve been frustrated and disappointed in many of them – they just didn’t do quite what I wanted, or they had an overwhelming amount of options that I didn’t need, or they were too expensive.

This year, I decided to stop being intimidated by the idea of building my own and made a very simple (okay, it wasn’t simple, but it looks simple when it’s on the site) “product box” plugin. I documented the entire process in a series of detailed TILCode posts, starting here. I learned a lot from making this, and – most importantly – I realized I’ve finally skilled up enough to take on a novel challenge and carry it through to completion.

You can see its repo here: https://github.com/MJGrant/amazin-wp-product-box and download it if you want to use it.

I completed five classes towards my CS degree!

I’m almost done! I’m on track to graduate in June of 2020.

I completed nearly 1/3rd of my degree this year. I took:

  • CS340 – Intro to Databases – basically “web dev, part 2” on a somewhat dated tech stack that didn’t go as deep into SQL as I’d hoped it would
  • CS344 – Operating Systems – a challenging class that deepened my understanding of bash, Linux, and sockets through four large projects
  • CS475 – Parallel Programming – a great class on an intimidating topic I’m not sure I would have studied on my own
  • CS372 – Intro to Networking – an incredibly detailed deep-dive into the inner-workings of the Internet, there’s no way I would have slogged through all those calculations on my own
  • CS362 – Software Engineering II – lots of automated testing practice that, unfortunately, left out a lot of standard industry tools and best practices

A common theme to the classes I took this year was “stuff I don’t have much experience with and probably wouldn’t have done on my own”.

One of the most valuable things about this degree program, to me, as someone who already has some professional programming experience, is when it exposes me to things I would not have found (or pushed myself through) on my own.

Some of these classes were very much outside my experience and quite challenging, particularly operating systems and networking. I appreciate learning about these things in a structured academic setting as opposed to on the fly at a job where people are counting on me to already know the basics.

I am still very happy with my choice to attend OSU. I’ve learned so much.

Here’s to a wonderful 2020!

OSU eCampus CS372 Intro to Networking review & recap

This post is part of an ongoing series recapping my experience in Oregon State University’s eCampus (online) post-baccalaureate Computer Science degree program. You can learn more about the program here.

Six-word summary: How the Internet Works: The Class

It’s an evenly-paced, heavy-workload class. It’s predictable (and the grading is sane), but it’s a lot of work and over half of your grade in the class comes from tests. When I think back to this class, I will remember the Canvas test interface. This class is quiz after quiz after quiz after quiz.

I’ve been in the OSU program since 2016, so I’m on the old “two electives” plan that treats CS372 as a required class. Going forward, though, I imagine most people reading this will be considering CS372 as an elective and wondering how it stacks up against other offerings.

CS372 Review

This class is just like CS271 (Assembly) in that it attempts to work you half to death in a subject that probably feels peripheral – if not completely unrelated – to your intended career path.

Every week feels exactly the same. There are about 1.5 hours of lectures to watch, a few book chapters to read, worksheets to do (if you want, they aren’t graded), a graded “summary exercise” you can take twice (they’re like quizzes in that they’re graded, but you get 6 hours to do them so they’re also open-book), and pick one or two of the following: a lab, a project, or a 90-minute quiz. That’s every single week from beginning to end.

You will study things like:

  • How a bit gets from one machine to another (in excruciating detail)
  • Calculate the realtime delays a packet will experience as it traverses a transmission medium
  • Different networking protocols
  • How things like TCP, UDP, HTTP, and email work
  • How delayed a packet will be (in milliseconds!) if the network is such-and-such congested
  • How wireless networking works
  • How a packet gets packaged and split up as it makes its way across the mighty interwebs
  • A brief glance at security-related topics

Networking as a topic is interesting and somewhat relevant to my web development interests but ultimately this class didn’t feel like it fed directly into my programming career or ambitions.

There’s very little programming in this class, and while I can imagine an interviewer or manager shrugging and saying, “Neat!” at the fact that I took a 10 week course in networking, I can’t imagine it’ll make or break my viability as a candidate for a programming job. (Just watch, now someone will ask me to label all the parts of a network-layer datagram header.)

Structurally, though, it’s a solid course – the workload was even, the grading was fair, and I walked away feeling like the class packed a lot of knowledge into my brain.

Class structure

  • 10 weeks
  • A weekly graded summary exercise of maybe 30 questions. You get two attempts, and they keep the higher score. There’s a 6 hour time limit but the most I ever spent was about 2.5 hours (and the fastest I ever got through one was about 45 minutes).
  • A lab every 2 weeks
  • 2 projects in which you program sockets using C or Java and Python
  • Midterm, final, both proctored and closed book but you get to take a one-sided note sheet in with you (size 4 font worked nicely)
  • There’s stuff due every week but it follows a consistent pattern

Time commitment

CS372 class was fairly time-consuming. Each week, I spent…

  • 2 to 4 hours on lectures and worksheets
  • 3 to 4 hours on the “weekly summaries” (6 hour graded quizzes), depending how many novel math problems were on them (this was worse in the first half of the quarter than it was in the latter half)
  • 3 hours on any given lab, if there was one that week
  • Anywhere from 6 to 10 hours on the current project, if there was one that week
  • 90 minutes on the quiz, if there was one that week
  • As many hours as I could afford studying for the midterm and the final in the 7 to 10 days leading up to the exam

Overall, I spent about 12 hours a week on this class. My children are in daycare 3 days a week to give me time to work on schoolwork, and I typically spent two of those days exclusively on this class’s material, plus some time in the evenings and the weekend if needed. The most time-intensive weeks were the ones that included a project. There were no “easy” weeks, and they don’t lighten the workload in anticipation of the midterm or final.

Compared to other classes, CS372 was a bit lighter than notoriously time-consuming classes such as CS162, CS261, and CS325. It was about on par with CS271 and CS361.

Lectures != worksheets != quizzes

Here’s my biggest gripe about this class, and the part where I try to help future students have a slightly easier time than I did.

The lectures lecture on one thing, the worksheets prompt you on another thing, and then the quizzes throw a bunch of math problems at you that weren’t really covered in the lectures or the worksheets. For many of them, you’re on your own to suss out the formula.

Worse, when you go Googling for help the top results are often for chegg.com, who will happily sell you the answers (eww) and old digital flashcards from previous students who posted answers but not how they actually solved the problems.

So, all that said, I thought I’d do something radical and document how to solve these problems for the benefit of anyone else taking this class.

How I solved…

Is it worth taking this class as an elective?

That’s the question everyone’s asking now as CS372 moves from being required coursework to an elective.

My short answer is no.

For me, there wasn’t enough practical problem-solving and programming in this course. The two socket programming projects are big and time-consuming, but they are dwarfed by the massive amounts of non-programming related work.

Some people may enjoy the theoretical nature of this class, but I prefer courses that teach skills and concepts I can immediately put to use on my software projects. I know there’s no “bad” knowledge, but I do feel like I spent 10 weeks studying something I will almost certainly never use.

OSU is rolling out a bunch of new electives in 2020 and beyond, and they all sound really exciting to me. (I’m graduating before I’ll have the chance to take them.)

Of the electives I have taken, I can easily recommend CS475 (Parallel Programming), and I will update this review with my opinion on CS492 (Mobile Dev) after I complete it next quarter (Winter 2020).

I really wanted to take CS493 (Cloud Application Development) but it requires CS372 as a pre-req and I’ll be graduating before Cloud runs again. Don’t be like me: many electives only run once or twice a year, so try to look ahead at least a full year so you can get into the electives you want. (I regret missing out on a chance to take Cloud.)

On the bright side, there’s no group work in CS372! So, if you’re sick of teammates who don’t produce anything and graded Canvas conversations, CS372 is a nice break from those sorts of things.

CS372’s Labs

This class alternates “labs” and “projects”. Labs are all about inspecting packets in a program called Wireshark, and projects are programming in a mix of C/Java and Python. (More on Projects in the next section).

There were 5 labs total and each one took about 3-4 hours apiece.

You can run the “trace” yourself, or you can use the downloaded trace from the author’s computer. I highly recommend using the downloaded trace. My own traces were polluted with tons of noise from my home network and I found that things were often “different” in one way or another. I wanted to just slice through labs like a hot knife through butter, and the easiest way to do what was using the provided trace data.

Each lab is 15-20 questions and you produce a pdf document that includes screenshots from Wireshark justifying your answer for each question. Overall, these were straightforward and manageable.

PS: I couldn’t get Wireshark to run on my Macbook so I had to use my Windows PC for the labs. Just a heads up for anyone else on a Mac. (And it’s well within the realm of possibility that the problem was unique to my machine or could have been solved with more effort.)

CS372’s Projects

There were two socket-programming projects in this class, and they were both challenging and time-consuming.

I felt like both of the projects were the most difficult part of the class. The rest of the class’s material is memorizing definitions and stepping through a series of steps to solve a math problem.

If you can, take CS344 first. The socket programming project in that class offers more hand-holding and provides some code you can adapt to CS372’s projects. (CS 372 students are directed to Beej’s Guide for help, but I didn’t find the guide useful at all. It’s long-winded, the examples are difficult to adapt to the project’s needs, and the attempts at humor are lame and eat up valuable screen real estate.)

I spent about 15 hours on Project 1 and about 30 hours on Project 2. Definitely start early on the projects and work in small, incremental steps. The times when I got mired were when I tried to do too much at once.

To the class’s credit, they provide a detailed rubric to test against. This was extremely helpful and something I wish all OSU CS classes would do.

Pro tip: The second project is meant to be a “portfolio piece”, meaning you can share your project code publicly once grading is over. Many students have done exactly this on GitHub. Part of being a software engineer is reading and understanding code you didn’t write, so there is value in looking up what others have done and stepping through it yourself until you comprehend it. My heartfelt thanks goes out to the past students who posted their Project 2 work publicly – I don’t think I could’ve gotten through the file transfer/sockets project without a few examples to study.

CS 372 Exams

Both the midterm and the final are proctored.

They are similar in structure to the weekly “summary exercises”, so taking the practice versions of those (look in the “Grades” section in Canvas and scroll all the way down for the ungraded practice versions) is time well spent. Expect a few novel problems that you’ve never seen before (those are definitely where I lost my points, some of the questions were unlike anything I’d studied).

My strategy (and I think this was a good one, as I walked away from this class with a solid A) was to score 100% or near it on every weekly summary, lab, project, etc. to make some room for mistakes on the tests.

Overall, I felt the tests were fair and that the materials provided were adequate preparation.

Final tips for CS 372

Get a good weekly routine going. The workload is extremely consistent (and rather heavy) so a consistent pattern will serve you well. Mine was something like…

  • Sunday and Monday were lectures and worksheets
  • Tuesday was the first attempt at the weekly summary and starting the lab or project assigned that week
  • Wednesday was all lab/project and the second attempt at the weekly summary
  • Friday was all lab and project
  • Saturday was anything that was left (I rarely needed Saturday, with the exception of Project 2, which totally dominated my life until it was over)

Get proficient (and fast) at solving the kinds of problems you encounter on the weekly summary exercises and you’ll be 80% of your way to a good grade on the exams.

Right before the midterm and the final, go back and practice the stuff you did a few weeks ago (preferably on a whiteboard or scratch paper like you’ll have in the exam). I was surprised at how rusty I got at problems I was solving with ease just a few weeks ago. You really want to be a well-oiled machine at solving the math problems come exam time.

Some whiteboard practice a few days before the final exam. I found it helpful to solve the same kind of problems over and over on the whiteboard so it was just muscle memory come test time.

Don’t be afraid to go straight to teh Googlez for help. The projects in this class basically leave you on your own, and the fastest and best way (imo) is to start Googling. You’ll do that at work someday, anyway.

Onwards to Winter 2020! I’ll be taking CS 492 Mobile next quarter, followed by Capstone in the spring.

OSU eCampus CS362 Software Engineering II review & recap

This post is part of an ongoing series recapping my experience in Oregon State University’s eCampus (online) post-baccalaureate Computer Science degree program. You can learn more about the program here.

Six-word summary: The nadir of OSU’s CS program

It’s not a great class. It’s not even a good one, unfortunately.

I have no idea how you dedicate 10 weeks to the topic of testing and spend about one sentence on the topic of Test Driven Development and absolutely zero time exposing students to any of the zillions of existing testing tools.

If you disliked CS290 for its obtuseness and CS361 for its general irrelevancy, I have bad news for you: CS362 is like they took the worst of those classes and combined them into one tedious, time-consuming course that doesn’t teach much that’s relevant to real world programming.

All this class does is have students write the same kinds of tests on the same (broken) code over and over and over until the end of the quarter. If you walk away from this class hating testing or unconvinced of its importance, please know that this class isn’t representative of what it’s like to write and maintain tests in a professional capacity.

CS362 Class Structure

  • 10 weeks
  • 5 code-writing assignments
  • Unproctored midterm and unproctored final
  • The occasional quiz, with no real consistency in length or content (most are 10 minutes of multiple choice but – surprise – one quiz is actually a standalone coding project, and you won’t know what you’re in for until you open the quiz!)
  • The way the class has you use Git/GitHub suggests they don’t understand how versioning works
  • Due dates and time allocations don’t follow much of a pattern
  • A large group project at the end where three students total form up a group and write more of the same tests
  • Very little that resembles “real world” testing practices

Repetitive assignments

Essentially, students spend the quarter doing five variations on the same project.

  • Assignment 1: Fork the class repo from GitHub and study the provided, bug-ridden Dominion code (this isn’t really an assignment in the way the ones that follow are).
  • Assignment 2: Refactor a small portion of the Dominion card code into separate methods (this part is basically copy/paste) and introduce five bugs of your choosing and document every change you make in a .pdf (the Dominion codebase was already packed with bugs, there was no reason to further break it)
  • Assignment 3: Write unit tests for your refactored Dominion card methods. Document your tests in a separate .pdf file. This assignment was an absolute beast in terms of scope and time required, and misconstrues unit tests as integration tests. Unit tests should test a small portion of the code’s behavior. These tests run the entire method and then verify that a variety of things either changed or did not change.
  • Assignment 4: Write “random tests” for your refactored Dominion card methods. This assignment was another beast and it basically repeats Assignment 3 except now you’re randomizing the inputs and running many more iterations.
  • Assignment 5: Test a classmate’s code with your tests. This assignment was unnecessary and painful. It essentially asks you to repeat Assignments 3 and 4 on a slightly different codebase (your teammate’s), which requires hours of tedious line-by-line updates to hundreds of lines of tests. The point of this assignment was to demonstrate that a change to the underlying codebase might require a change to the tests, but they could have made that point with far less busywork.
  • Group Assignment Part A: The instructor hand-picked 11 bugs from the “Find Dominion Bugs” Piazza thread that everyone wrote hundreds of bugs into earlier in the quarter. Students form up into groups of 3 and produce a .pdf documenting the ways they plan to capture those 11 bugs in a collection of “unit” tests.
  • Group Assignment Part B: Actually fix the bugs identified in Part A. Commit the functioning test code to the team’s shared GitHub repo. Turn in 40+ pages of documentation on every single bug, fix, and decision made.

You don’t get to fix any bugs until the very end of the class, and you only get to fix a few. It’s very unsatisfying to write endless tests without fixing anything. I fixed things to see if my tests were any good, but I had to put the code back to its broken state before turn-in. There’s no sense of progression.

Also, the numbering and due dates don’t follow a strict pattern so double check those due dates.

The class’s git practices are a mess

This class does more with Git/GitHub than any other OSU class, but it doesn’t teach it nor does it do much to help students who are struggling with it.

If you are new to git, do yourself a huge favor and get used to it before you get to this class. Use it for classwork and personal projects. GitHub lets you make private repos now. This class won’t teach git, but it will expect you to be proficient enough to fork a repo, make branches, and push/pull to them, and you’ll have to do it in a group setting by the second half of the course, so start getting familiar with it now. Don’t burden your group mates with teaching it to you (or fixing your mess) at the 11th hour.

There was one particular “bad practice” I think this class could improve. Throughout the course, the assignments instruct students to make multiple copies of the “dominion” folder, which is weird and unnecessary in a versioning system.

By the end of the quarter, my repo looked like this:

  • dominion/ <– pristine untouched copy of dominion game code
  • projects/
    • robertwen/ <– the teacher’s folder, untouched in my repo
      • dominion/ <– another untouched copy
    • granaman/ <– me
      • dominion/ <– where my work was done
      • FinalProject-BugFree/
        • dominion/ <– group tests and bug fixes
      • FinalProject-Bugs/
        • dominion <– group tests, no bug fixes

There is no reason for the dominion folder to exist in 5 different places in the project repo. Two of those versions are completely untouched throughout the class.

Branches and pull requests could have handled versioning for each assignment, and if anyone needed to refer back to the original code it was always available in the teacher’s repo (the one we all forked our copies from) or by checking out an earlier version of the repo.

I think this poor organization sets a bad example for students, many of whom are still new to git and were confused as to why the same directory was copied into multiple locations.

No feedback on your work

Unfortunately, it’s rare to get feedback on your code in this program in general. CS362 is not unique in that the TAs don’t give suggestions for improving your coding.

But this lack of feedback is exceptionally disappointing in a class that’s has you write literally hundreds of lines of tests of your own design only to hear NOTHING back.

Students are given a lot of freedom in this class, but they’ll never know if they were on the right track or not. I wrote a ton of code in this class and sent it into a void, never to be heard from again. I got an A on most of it, but maybe that’s because the accompanying document was 30 pages long.

The one time I didn’t get an A on an assignment was because the TA marked me down 15% for exposing a crash bug that was in the dominion code to begin with. At that point I didn’t know what was real anymore.

If we aren’t writing tests to expose bugs then why ARE we writing them?

I don’t know.

But I can tell you what I didn’t do in the assignments that built on that assignment: I didn’t expose that crash bug again.

I let that bug sit there, undocumented and unacknowledged, in all future assignments because to test it would be to turn in a “crash bug” that would deduct 15% off my grade each time.

I’ve worked on software for over a decade and the idea of ignoring a crash bug and knowingly failing to cover it with a test is ridiculous. Like, fireable offense ridiculous. Nobody who takes pride in their work or values transparency does this. I died a little bit inside.

But that’s the kind of class CS362 is. :(

Testing is misrepresented

If you’re hoping for help with coding style, theory, practical approaches, “what to do vs. what not to do”, you won’t find it here. What this class calls “unit testing” is more like integration testing, and little effort is made to explain the difference.

There is no TA feedback on work produced in this class. Again, that’s not a surprise, but the work in this class seems designed to invite feedback, especially since the assignments are just variations on each other. Surely the TA looked at my code and had at least one idea for improving it? I don’t think I’ve ever been in a code review of a substantial amount of code where literally nothing was suggested.

Every assignment is accompanied by pages (often 20 or more) of written documentation. There is no need to write this much documentation on code that is, for the most part, self-evident in what it is doing. I mean, there’s a string baked into every test announcing its purpose – how much more help does a TA need to figure out what the code is doing? Plus, nobody writes (or reads) this kind of stuff in the real world. They expect your code to be self-documenting and, if a process does have to be documented, it’s expected to be concise, not bloated for length’s sake.

This class doesn’t mention any existing industry-standard testing libraries and tools. I thought they could at least mention test runners, configuring tests to run automatically as part of a deployment pipeline, mocking data and services for tests, and in a 10 week class there’s time to really dig into at least one or two assert libraries.

This class has students write tests in the style of integration tests but calls them unit tests. This just made me cranky – a unit test tests a very small portion of the code. Like, a unit test tests a method that only does one job. The methods we tested were larger than that, like 25-100 lines long, and made multiple changes to the game state via their own code and calls to other methods. That’s more of an integration test at that point.

At least I wasn’t alone, I guess?

CS362 Group work

The last two assignments require students to work in self-selected groups of 3 to write unit tests and fix bugs (identified and assigned by the instructor) in the Dominion code.

There is a special place in software developer hell for the group member who dumped their unreviewed code into our repo hours before the project was due and then went out to dinner. This isn’t unique to CS362 but it’s ridiculous the way every other group in this CS program contains a student who just doesn’t give a shit. This student promises everything and produces nothing, turns in work hours before the deadline, and acts like it’s no big deal that their work is garbage.

In CS362, they split the last project into two parts (A and B), made B depend on the work done in A, but gave them both the exact same due date. So, of course, one of our teammates saved all of their work on A and B for the last day and turned in a bunch of grade-dropping junk at the 11th hour.

The very least they could do in CS362 is make it so Part A is due before Part B so the functioning members of the team can identify the non-performing team member earlier than the day it’s all due.

What’s missing from CS362

Test-driven development. This class could’ve given students a simple project and asked them to write and turn in the tests before writing the project code. If they still want students to write reams of documentation, an assessment of how well those tests “held up” once the program was written might be valuable.

Expose students to at least one existing test/assert library. There are so many to pick from. Pick anything, it doesn’t matter, the point is to show students that test libraries exist and get them accustomed to writing in their unique syntax. My first assert library was Chai: https://www.chaijs.com/api/assert/ and I named (and structured) my C test methods for CS362 similarly.

Teach more testing know-how. The class could cover mocks and stubs, unit and integration tests, and how to write a bug. The class tries to cover “how to write a bug” but the only thing you’re really graded on is how well you followed the bug reporting template they provided, so don’t leave off irrelevant fields (such as “browser version” for a program that runs as a standalone executable) or you’ll lose points.

Show students how testing can be part of an automated build/deploy pipeline. Testing in this class kinda comes off as a thing you do locally on your machine when/if you modify the codebase and then forget about. It kinda blew my mind when I started working to see that tests were actually the build’s gatekeeper – that is, the deploy didn’t happen if the automated tests didn’t pass, and the tests were run in their own environment and occasionally subject to failures that didn’t happen on our dev machines. This topic could be its own post, but it might be a nice thing to expose students to.

Pass on some testing “wisdom”. One of my favorites is “if you haven’t seen it fail, then you haven’t really seen it pass“. That is, if your test passes right out the gate, don’t move on right away – perhaps it always passes?

But don’t just take my word for it…

If you’re used to checking the Course Explorer to scope out a class ahead of time, be warned that most of the reviews for CS362 are from a previous version of the class. They’re not accurate anymore. In the meantime, here is some real (anonymized) commentary from the class Slack chat.

Pretty much.

There were better ways to spend $2000. But it had to be done, and I’m glad it’s over. I hope that OSU improves this course soon.

Onwards to CS 492 in Winter 2020!

Networking – how to find the netmask, network address, host mask, broadcast address, number of possible hosts, and host number for a given IP address

Suppose you are given an IPv4 CIDR address

153.10.22.56 /22 

This IP address represents a particular device (“host”, in networking terminology) on a network, such as a laptop computer or gaming system.

From this simple set of numbers you can calculate all sorts of things about the network it belongs to, such as the network’s starting address, the number of devices (“hosts”, in networking terminology) the network supports, and the ordinal number in the network of this particular device. Let’s go be leet hackers…

Find the Network mask

The number after the “/” indicates what portion of the network address belongs to the network itself, and which portion is available for hosts. In the case of /22, it means the first 22 bits are used for the network itself, and the remaining 10 bits can be used by the various hosts that join the network.

In a 32-bit binary number, the first 22 bits should be 1’s, and the rest 0’s.

11111111 11111111 11111100 00000000 // that's 22 1's

Convert each “chunk” back to decimal and you get:

255.255.252.0  // the netmask! 

Find the Network’s Address

Now that we know the network mask, we can figure out the network’s address. The network address is the address of the network itself and it is the lowest address on the network.

If you’ve ever poked around your own home network, you might recognize 192.168.1.0 (or similar). All devices that join the network are given a similar, higher-numbered IP address when they are on the network: your phone might be 192.168.1.05, your laptop might be 192.168.1.12, etc.

You can find the network’s address by using your knowledge of the IP address of any given device on the network and the netmask.

Start by converting the IP address and the netmask to binary. Then, perform a bitwise and, meaning that for any position that has two 1’s, the result will be a 1. (Two zeroes or a 1 and a zero results in a 0.)

153.10.22.56        10011001 00001010 00010110 00111000
255.255.252.0       11111111 11111111 11111100 00000000
                    --------------------------------------
Bitwise AND:        10011001 00001010 00010100 00000000

Convert the result of the bitwise and back to decimal to get your network address:

153.10.20.0        // the network address!

Sanity check: The network address is always the lowest address on the network, so it makes sense that a device found at 153.10.22.56 could be somewhere in a network that began its addresses at 153.10.20.0. You should be suspicious of your result if you end up with a network address that is higher than the IP you were given to start with.

Find the Host Mask

The host mask is the inverse of the network mask and we’ll use it later on to find the broadcast address.

Go back to your network mask result in binary and flip the 1’s and 0’s to get the host mask:

11111111 11111111 11111100 00000000 // network mask
00000000 00000000 00000011 11111111 // host mask (bits flipped)

You may wish to convert it to decimal:

0.0.3.255

We’ll use the host mask in the next section to find the broadcast address.

Sanity check: Since the host mask is just the opposite of the network mask, you might able to “eyeball it” by finding the difference between each portion of the address and 255. For the given netmask, 255.255.252.0, you could subtract 255-255 = 0 for the first two parts, 255-253 = 3 for the second-to-last part, and 0-255=|-255| for the last part. Be suspicious of any result you get that doesn’t appear to be the “inverse” of the net mask you started with.

Find the Broadcast Address

The broadcast address is the highest address on the network.

To find it, use the IP address you started with and the host mask from the previous step. Perform a bit mask on them, meaning that anywhere you have a 1 is good enough to get a 1 in the result. Two 1’s = 1 and a 0 and a 1 = 1.

153.10.22.56        10011001 00001010 00010110 00111000  // IP 
   0.0.3.255        00000000 00000000 00000011 11111111  // host mask
                    --------------------------------------
Bit mask:           10011001 00001010 00010111 11111111  // broadcast addr.

Convert to the broadcast address to decimal:

153.10.23.255  // the broadcast address!

Find the number of possible hosts

To find the maximum number of hosts a network can support, go back to that “/22” from the IP address you started with. Subtract it from 32. (Why 32? Because that’s how many bits are in an address, total.)

32 - 22 = 10

Now raise 2 to that power:

2^10 = 1024

And subtract 2:

1024 - 2 = 1022  // hopefully that's enough room for everyone 

We subtract 2 because there are two reserved host numbers: the network address itself, and the broadcast address. We can’t assign those to laptops and smart TVs so we don’t include them in the maximum number of supported hosts.

Find the host number

My class often asked us for the “host number” of a device but didn’t really explain what was meant by that or how to calculate it.

The host number is just the ordinal number of the host on the network. It’s the Nth device on the network. Pretend you’ve already got a printer, a TV, and a computer on your network. You add your phone. Your phone is the 4th device to join. Its host number is 4.

In other words, you’re not looking for an IP address here, you’re looking for an ordinal number. For the sake of questions like the ones I got in CS372, you’re also assuming that hosts were added “in order” and assigned their IP addresses accordingly.

You need to know both the network’s address and the IP of the device itself. it from the IP if you know where the network starts.

153.10.22.56   // this is where we are, what is our host number?
153.10.20.0    // this is where the network starts

I imagine each section of the IP address to be like a bucket. Each bucket holds 255. Once a 256th is added, the number to its left has to increase by 1 and that bucket is reset to 0. (This is just like “carrying over” a number the way we’ve done since elementary school.)

Let’s do an easy one first.

153.10.20.0   // this is where the network starts
153.10.20.1   // what host number am I?

The device at 153.10.20.1 is host number 1. It’s the first host on the network because 153.10.20.0 is reserved as the network’s own address.

Here’s a harder one:

153.10.20.0    // this is where the network starts
153.10.20.255  // what host number am I?

The device at 153.10.20.255 is host number 255.

Now let’s see what happens when the number to the left also changed.

153.10.20.0    // this is where the network starts
153.10.21.1   // what host number am I?

Hey, that 20 became a 21! We must have added a lot of hosts to this network. We added so many, that the 153.10.20.nnn space was exhausted and we had to increase 20 to 21 and start numbering over. This “carry over” is just like you did in elementary school math, except here the limit is 255.

Since this is where it gets complicated, I’ll show you the steps:

153.10.20.0    // this is where the network starts
153.10.21.1 // what host number am I?
20 - 21 = 1    // there was 1 "carry over" 
1 * 255 = 255  // which means 255 hosts were added...
255 + 1 = 256  // and there's still 1 in the far-right bucket
256 + 1 = 257  // add that 1 from the first line to account for reserved addresses

The device at 153.10.21.1 is host number 257.

Hopefully you see the pattern here: each section is like a “bucket” that can be filled to 255, and then it has to “carry over” to the left and get reset to 0.

Now that you know how it’s calculated, we can look at “reverse engineering” it from our problem’s IP address.

153.10.20.0    // this is where the network started
153.10.22.56   // this is where we are, what is our host number? 
22 - 20 = 2     // two "carry overs" in the bucket
2 * 255 = 510   // 510 were added to get from 20.0 to 22.0
510 + 56 = 566  // add the 56 from the last bucket
566 + 2 = 568   // add 2 from the first line

The device at 153.10.22.56 is host number 568.

I think that last step is worth elaborating on a bit more. The 2 was added because 153.10.21.0 and 153.10.22.0 are also not used for devices. The 2 was derived from subtracting 20 from 22 (the second-to-last segment of the IP address).

More host number examples

Example 1:

128.193.42.0     // this is where the network starts
128.193.43.35    // this is where we are, what is our host number? 
43 - 42 = 1      // there was one "carry over" 
1 * 255 = 255    // so we know at least that many hosts were added so far
255 + 35 = 290   // add the 35 from the furthest-right bucket
290 + 1 = 291    // add that 1 from the first step, we're host #291

Example 2:

128.193.0.0      // this is where the network starts
128.193.43.35    // this is where we are, what is our host number?
43 - 0 = 43          // that's a lotta "carry overs"
43 * 255 = 10965
10965 + 35 = 11000
11000 + 43 = 11043   // we're host #11043

Networking – how to solve the “find the utilization for a TCP sender that is continuously sending” problem

In this post: A detailed, step-by-step guide demonstrating the steps I use to solve a problem similar to one encountered in my CS 372 Intro to Networking class.

You have a TCP sender that is continuously sending a 1,096-byte segment. If a TCP receiver advertises a window size of 8,551 bytes, and with a link transmission rate 35 Mbps an end-to-end propagation delay of 33.3 ms, what is the utilization?

(Assume no errors, no processing or queueing delay, and ACKs transmit instantly. Also assume the sender will not transmit a non-full segment.)

Suppose they want the answer as a percentage rounded to one decimal place.

This one breaks down into a series of simple steps:

Step 1: Calculate how many segments the pipeline holds

formula: 
segments = floor(pipeline size / segment size)
8551 / 1096 = 7.802 
= 7 segments

Remember to round down! There are no partial segments in this pipeline.

Step 2: Calculate how long it takes to send one segment

This uses the L/R formula, which is:

formula: 
time to send one segment = length of segment / transmission rate

(Don’t forget to use the same units for both. I convert everything to bits for consistency.)

(1096 * 8) / (35 x 10^6) = 0.250514 time to send one segment

Step 3: Calculate the Round Trip Time (RTT)

formula: 
RTT = propagation delay * 2 
33.3 * 2 = 66.6 round trip time

Step 4: Calculate the delay per packet

formula: 
delay per packet = RTT + time to send one
66.6 + 0.250514 = 66.850514 delay per packet

Step 5: Calculate the utilization!

formula: 
utilization = (segments * time to send one) / delay per packet
(7 * 0.250514) / 66.850514 = 0.026316

Move the decimal to places to the right to get the percentage.

= 2.6% utilization

Networking – how to solve the “effective delay” problem

In this post: A detailed, step-by-step guide demonstrating the steps I use to solve a problem similar to one encountered in my CS 372 Intro to Networking class.

Given an effective delay of 99ms when network usage is 76%, what is the effective delay when network usage is 53%?

Suppose they want the answer in milliseconds rounded to 1 decimal place.

You might’ve seen a variation of this problem where the delay is given in ms and the network usage is 0%. If you’re given a delay with a network that’s in-use, you have to first figure out what the the delay would be if the network congestion was 0%.

Let’s call that value x and solve for it.

We will use the same D = Do/(1-U) formula we would use if we already knew the delay for a zero-congestion network, but we will put our 99 ms delay where D goes and our unknown “x” value for Do.

Do / (1 - U) = D
x / (1- 0.76) = 99ms 
x / (0.24) = 99

Multiply both sides by 0.24 to turn that into a value for x:

x = 0.24 * 99
x = 23.76 <-- network delay in ms when congestion is at 0%

Now that we know the delay in ms when congestion is at 0%, we can plug it into the D = Do/(1-U) formula again.

D = 23.76 / (1 - .53) 
D = 50.6 ms

It’s not a difficult problem but my class’s materials only covered the “easy” version so I wanted to demonstrate the “missing steps” for anyone else who might be wondering what to do when they give you the delay for a partially-congested network instead.

Networking – how to calculate the size of MSS for a “mythical set of protocols”

In this post: A detailed, step-by-step guide demonstrating the steps I use to solve a problem similar to one encountered in my CS 372 Intro to Networking class.

Imagine a mythical set of protocols with the following details:

  • Maximum Link-Layer data frame: 1,331 bytes
  • Network-Layer header size: 29 bytes
  • Transport-Layer header size: 34 bytes

What is the size, in bytes, of the MSS?

Answer:

1331 - 29 - 34 = 1268

You just subtract the header sizes from the data frame. Easy.

You probably could’ve guessed that, but if you’re in CS372 and you’re reading this then it might be because you weren’t 100% sure wanted to check your work but the class materials didn’t really cover it. And that’s the thing that irked me about this problem: the class just threw it at us as a surprise on the weekly (graded but open-book) exercises in week 6. Then you go a-Googlin’ and end up somewhere like chegg.com who wants $15 to sell you a solution! That’s just silly.

Now you know: calculating the maximum segment size is just a simple subtraction problem.

Networking – how to solve the “non-persistent vs persistent HTTP” problem

In this post: A detailed, step-by-step guide demonstrating the steps I use to solve a problem similar to one encountered in my CS 372 Intro to Networking class.

A client’s browser sends an HTTP request to a website. The website responds with a handshake and sets up a TCP connection. The connection setup takes 2.5 ms, including the RTT. The browser then sends the request for the website’s index file. The index file references 2 additional images, which are to be requested/downloaded by the client’s browser. How much longer would non-persistent HTTP take than persistent HTTP?

Suppose they want the answer in milliseconds rounded to 2 decimal places.

You can get fancy with writing out each and every request (and this is what they’ll show in the worksheet), but there’s a faster way, especially if the number of referenced objects is more than a few.

Step 1: Count the total number things you’re getting, ie: 1 index file + 3 images = 3. That’s your N value.

index.html + image1.jpg + image2.jpg = 3 things total

Step 2: For non-persistent HTTP, your answer will be the result of N x 2.

3 x 2 = 6 connections needed for non-persistent HTTP

(That’s because for each and every item you request, you also need to request a TCP connection.)

  1. TCP request
  2. index.htm request
  3. TCP request
  4. image1.jpg request
  5. TCP request
  6. image2.jpg request

Step 3: For persistent HTTP, do N+1

3 + 1 = 4 connections needed for persistent HTTP

(That’s because you only need to connect once, and then request all the things you want.)

  1. TCP request
  2. index.htm request
  3. image1.jpg request
  4. image2.jpg request

Step 4: Now you have:

6 connection requests for non-persistent HTTP
4 connection requests for persistent HTTP

Step 5: Find the difference.

6 - 4 = 2

Non-persistent HTTP needs 2 more connections than persistent.

Step 6: Multiply that difference by how long each connection takes to set up.

2 x 2.5 milliseconds = 5 milliseconds. 

Double-check your rounding and units! You might be asked to give this in seconds instead of milliseconds.

In summary

  • Sum the things (index file and images) to get N
  • Non-persistent HTTP = N x 2
  • Persistent HTTP = N + 1
  • Find the difference
  • Multiply the difference by how long a round trip takes

Networking – how to solve the “fileX and fileY” problem

In this post: A detailed, step-by-step guide demonstrating the steps I use to solve a problem similar to one encountered in my CS 372 Intro to Networking class. 

You’re given a link with a maximum transmission rate of 70.5 Mbps. Two computers, X and Y, want to transmit starting at the same time. Computer X sends File X (18 MiB) and computer Y sends File Y (130 KiB), both starting at time t = 0.

  • Statistical multiplexing is used, with details as follows
    • Packet Payload Size = 1000 Bytes
    • Packet Header Size = 24 Bytes (overhead)
  • Ignore Processing and Queueing delays
  • Assume partial packets (packets consisting of less than 1000 Bytes of data) are padded so that they are the same size as full packets.
  • Assume continuous alternating-packet transmission.
  • Computer X gets the transmission medium first.

At what time (t = ?) would File X finish transmitting?

Suppose they want the answer in seconds (hah – not milliseconds like usual!) and they want it rounded to two decimal places.

The “twist” of this problem is that File X and File Y take turns being sent in little chunks of 1000 bytes (plus 24 bytes of overhead).

First a piece of X goes, then a piece of Y, then back to X, etc.

Y is smaller so you run out of Y before you run out of X, but fortunately for us, this problem wants to know when X is done being sent. So really what we’re looking for is how long it takes to send both X and Y.

Here’s a drawing I made to visualize the problem (obviously, X and Y are both way more than just 10 and 5 packets in our real problem).

Diagram depicting X1, Y1, X2, Y2, X3, Y3, X4, Y4, X5, Y5, X6, X7, X8, X9, X10 interwoven with each other. Each block represents a packet.

Step 1: Figure out how many packets have to be sent, total

Our packet size (1000 bytes) and our overhead size (24 Bytes) are given in bytes, so I’m going to turn the two filesizes into Bytes instead of bits for this one. (We can always turn it into bits later if needed.)

File X = 18 MiB // turn to bytes by multiplying by 1024 twice
= 18 * 1024 * 1024 
= 18874368 bytes
File Y = 130 KiB // turn to bytes by multiplying by 1024 once 
= 130 * 1024
= 133120 bytes

Now X and Y are in the same units (bytes) and we can use that to figure out how many packets of 1000 bytes each one would be divided into.

File X = 18874368 Bytes / 1000 = 18874.368 packets

Our result isn’t a whole number. What should we do with the “partial packet”? Well, the problem tells us that any partial packets should be padded up to a full size. In practice, this just means we round up to the next nearest whole number. 18874.368 becomes 18875 packets for File X.

Now do the same for File Y:

File Y = 133120 Bytes / 1000 = 133.12 
Round up to get 134 packets for File Y

(Save these numbers for later, we’ll come back to them.)

Step 2: Calculate the time needed to send one packet plus its overhead

Add the packet size and the overhead size (both must be in the same units; here, that’s bytes but your variation of this problem may differ). Since transmission rate is in bits, let’s also multiply the packet size + overhead size by 8 to put it in bits, and multiply the transmission rate by 10^6 to put it in bits also. That looks like…

L/R, where R's units is what the outcome of this calculation will be 
= (1000 Bytes + 24 Bytes * 8) / (70.5 Mbps x 10^6)
= (8192) / (70.5 x 10^6)
= 0.0001161985816 seconds // same units as R (bits per second) 
= 0.11619858156 msec // multiply by 1000 to get milliseconds

We now know that it takes 0.11619858156 milliseconds to send one packet (and its overhead). Save this number for later.

Step 3: Calculate the time it takes to send all packets

This step is easy: we know how many packets we have between File X and File Y, and we know how long it takes to send one of them, so let’s figure out how long it takes to send all of them.

18875 packets + 134 packets = 19009 packets to send
19009 packets x 0.11619868156 msec = 2208.820738 milliseconds to send all 

Step 4: Convert to seconds and round to 2 decimal places to get the final answer

This variation of the problem wants the solution in seconds, so divide our result from Step 3 by 1000 to get 2.208820738 seconds and round it to two decimal places to get 2.21 seconds, which is when time at which File X will be done sending.

What if it wanted to know when the smaller file was done sending?

Well, let’s look at our diagram again. Here we can see that for every packet of Y sent, a packet of X has already been sent.

For every packet of Y that is sent, one from X precedes it. In other words, to send all 5 packets of Y we must also send 5 packets of X.

We know Y is made up of 134 packets, so that many packets of X will also be sent. That means for our final answer all we have to do is figure out how long it’ll take to send 134*2 packets.

134 * 2 = 268 // all of Y + same amount of X
268 packets x 0.11619868156 msec = 31.14124666 msec to send File Y

Convert that answer to seconds and there we have it: 0.03 seconds have elapsed when File Y (the smaller file) is done sending.

Networking – how to solve the “routers in a sequence” problem

In this post: A detailed, step-by-step guide demonstrating the steps I use to solve a problem similar to one encountered in my CS 372 Intro to Networking class. Yeah, I’m still cranky that this class makes no effort to teach this stuff. They just send you a-Googlin’. Welcome, Googlers…

Suppose there are 3 routers in sequence between Host A and Host B, all of which use store-and-forward routing. What is the total end-to-end delay for a packet originating from Host A with destination Host B, under the following conditions.

  • Each of the link transmission rates are 5.1 Mbps
  • The total distance from Host A to Host B along its path of transmission is 175.1 km
  • The speed of propagation through the transmission medium is is 2.7 x 108 m/s
  • The packet size is 2 KiB

Remember that you must also uplink from Host A to the first router. Suppose this one wants the answer in milliseconds rounded to 1 decimal place.

Step 1: Figure out the transmission delay for one link

The “twist” of this problem is that there are multiple transmission delays to account for. Every “hop” incurs a transmission delay.

We can use the L/R formula with the known packet size (2 KiB and known transmission rate (5.1 Mbps) to calculate the transmission delay for one node to another. Node here means host or router. Be sure to convert both into bits.

(2 KiB * 8192) / (5.1 Mbps * 10^6) 
= 16,384/5,100,000 = 0.00321254... seconds

Remember that the units R is in determines the outcome, so this result is in seconds. Multiply it by 1000 to turn it into milliseconds and save this result for later.

0.00321254 x 1000 = 3.21254901... milliseconds

Step 2: Figure out the entire path’s transmission delay

Recall that our three routers and two hosts basically look like this:

[A]—[1]—[2]—[3]—[B]

We know the transmission delay (the time it takes for a bit to be placed on the transmission medium) at one host or router, and we know how many routers there are total, so we can figure out how many “transmission delays” our bits will go through by multiplying the total number of routers plus one (to account for being placed on the transmission medium by A) by the transmission delay we calculated in step 1.

(R + 1) * 3.21254901  // R is the number of routers
= (3 + 1) * 3.21254901 // add 1 to account for initial link
= 4 * 3.21254901 
= 12.85019604 milliseconds

This number represents the total amount of transmission delay experienced by bits traversing this multi-router path.

Step 3: Figure out the propagation delay

Recall that our distance is 175.1 km and our speed of propagation is 2.7 x 108 m/s and the propagation delay formula is Tprop = d/s. Note that km has to be turned into meters and the result will be in seconds.

= (175.1 * 1000) / (2.7 x 108 m/s) = 0.0006485185... seconds 
0.0006485185... seconds x 1000 = 0.64851 milliseconds

Step 4: Add total trans. delay + propagation delay

 12.85019604 + 0.64851 = 13.49870604 milliseconds

And that’s it, you’re done! Hope you found this helpful!