Source Maps – Debug your Bundled, Minified Code

While developing a web app, it’s a good idea to bundle all your various JavaScript files and libraries into a single, minified file for production. Bundling makes your app faster by reducing download size and saving the browser from having to perform multitudes of HTTP requests just to render the page.

However, bundling comes at a cost. Debugging bundled JavaScript is often a nightmare. If your app has an error, the error in the console points at the point in the bundle file where the code failed, which may or may not bear any resemblance to your actual project files.

Which would you rather debug?

Which would you rather debug?

Wouldn’t it be great if the debugger was smart enough to point you at your original, pre-bundled code instead? Enter source maps, a way to preserve reference to your original file and code structure, even through bundling.

Before We Begin

This article uses the following tools, so make sure you have them first (or at least the basic concepts about them):

(PS: Grunt and Browserify can do source maps, too.)

How Source Maps Work

A source map is a composed of data that links pieces of your minified bundle back to the original, unbuilt files. Modern dev tools will automatically follow the map for any console.log or line queries you perform.

Showing The Problem

Let’s say I have a simple console.log in an angularjs controller someplace:

angular.module('myApp')
  .controller('CtrlMain', function ($scope) {
    console.log('hello');
  });

When we run the app and look in Chrome’s JavaScript console, it provides a very unhelpful link to where the log was executed:
Screenshot from 2014-11-27 17:07:20Oh, it’s in the one and only line in my bundled script? I never would have guessed.

To generate bundle.js, I’m using the following task in Gulp:

gulp.task('build-js', function () {
  gulp.src([jsDir + 'module.js',jsDir + '*.js'])
    .pipe(concat('bundle.js'))
    .pipe(ngAnnotate())
    .pipe(uglify())
    .pipe(gulp.dest('build'));
});

This concatenates all JavaScript files into a single file called bundle.js, creates dependency injection annotations so Angular doesn’t choke on the minified code, minifies the code using gulp-uglify, and finally writes it to a build directory.

Adding Source Maps To The Build Process

Here’s how to add source map creation to this build process, using gulp-sourcemaps:

gulp.task('build-js', function () {
  gulp.src([jsDir + 'module.js',jsDir + '*.js'])
    .pipe(sourcemaps.init())
    .pipe(concat('bundle.js'))
    .pipe(ngAnnotate())
    .pipe(uglify())
    .pipe(sourcemaps.write())
    .pipe(gulp.dest('build'));
});

Before we do anything to the JavaScript files, we call sourcemaps.init(). This tells gulp-sourcemaps to pay attention and generate our source maps while the build process occurs. sourcemaps.write() signals that we’re done processing the .js files and ready to write out the bundled file.

IMPORTANT NOTE: All plugins used between sourcemaps.init() and sourcemaps.write() must be supported specifically by gulp-sourcemaps. You can find a list of supported plugins here.

The Final Product

Now let’s build and run the app again and see what we get:

Screenshot from 2014-11-27 18:19:39

Much better

Now the console.log behaves as if we’re working with the clear, unbuilt versions of our files. We can click that link on the left to see exactly which line contains the log. Ahhhh.

 

7 IntelliJ IDEA tips and settings you’ll probably love

Hardly a week goes by where I don’t learn something new about IntelliJ! Here are just a few of my favorite IntelliJ settings tweaks and workflow tips.

This article was written for IntelliJ IDEA 14 professional and community editions.

1. Exclude some folders from search

If searching in your project is returning a lot of non-project files you’d rather not see (such as files in /node_modules, /tmp, or /build), you can permanently exclude those files.

Go to File > Project Structure > Project Settings > Modules. Right click any folders you don’t want to ever search in and mark them as Excluded. Done!

exclude_folders_from_search_intellij_idea

2. Highlight current file in project hierarchy

Do you want to see where your currently active file is in the project structure? Of course you do. Turn on “Autoscroll from Source” by clicking the little gear in the little toolbar at the top of the project structure window.

Now whatever file you’re editing will be automatically highlighted in the project structure pane.

highlight_file_in_hierarchy

3. Start a local server with a single click

For all your little web projects that you want to test in browser without bothering to set up a server first, just right click the file you want to serve and choose Open in Browser.

local_server_intellij_right_click_file

I overlooked this feature in the past because I thought it was just going to open the file all vanilla and stuff, but it totally starts a little standalone server! 

intellij_temp_server

4. Resolve git conflicts

IntelliJ IDEA’s git merge conflict resolution tool is fun and convenient. Just right click in any of your project’s files (doesn’t have to be the one with the conflict in it) and go to Git > Resolve Conflicts…

right_click_resolve_git_merge

You’ll get a modal with a list of all the files containing conflicts. Choose Merge…

files_with_conflicts_git_merge_intellij

In this screen, you can compare your changes (on the left) to what you got from the server when you pulled (on the right) and view the result (in the middle). Move changes to the result by clicking on the >> and << symbols, and reject changes by clicking the X. You can even edit the final version manually right there in the middle pane.

intellij_merge_conflict_tool

5. Search EVERYWHERE

You know you’ve seen it… somewhere. But your project is huge and that file could be anywhere. Double-tap the Shift key to open up Search everywhere. I use this shortcut to hop between files all the time because it’s just so convenient.

6. GitHub integration

In the right click menu (click inside a file or on a file in the project hierarchy) there are a couple shortcuts to GitHub. Open on GitHub is handy for jumping right to the file you’re working on to view its history, Create Gist is handy if you like making Gists.

create_gist_from_intelliJ

7. Turn on Annotations to know who to blame (it was probably you)

Who wrote this line of code? Now you can know: right click in the left margin (where the line numbers are) and turn on Annotate.

annotate_intellij

Hovering your cursor over individual lines reveals the last commit message, too.

git_annotate_names

There you have it: a short ‘n sweet guide to 7 helpful IntelliJ tools, shortcuts, and settings.

How to fix “Failed to retrieve directory listing” in FileZilla 3.10 with Bluehost

2018 update: change your connection protocol to SFTP – SSH File Transfer and it should work.

This post is 4+ years old and describes a very old problem yet it continues to draw visitors experiencing this problem (or similar). The method it describes is not secure by modern standards. You should update to the latest Filezilla and use SFTP.

Here’s a dumb thing I lost 2 hours on today: the new FileZilla (version 3.10) uses FTP over TLS by default, but not all hosts support this (like mine).

In my case, I was trying to log into my Bluehost account via FileZilla, which I hadn’t done in a month+ while I was moving and changing ISPs, so that made it even more difficult to narrow down the cause of this error.

Status: Resolving address of ftp.mywebsitename.org
Status: Connecting to 11.222.333.444:21...
Status: Connection established, waiting for welcome message...
Status: Initializing TLS...
Status: Verifying certificate...
Status: TLS connection established.
Status: Server does not support non-ASCII characters.
Status: Connected
Status: Retrieving directory listing...
Command: PWD
Response: 257 "/" is your current location
Command: TYPE I
Response: 200 TYPE is now 8-bit binary
Command: PASV
Response: 227 Entering Passive Mode (11,222,333,444,167,111)
Command: MLSD
Error: Connection timed out
Error: Failed to retrieve directory listing

By far the fastest, easiest fix is to uninstall FileZilla 3.10 and go back to an earlier version. I went back to FileZilla version 3.7.3 and I was finally able to get some work done.




It’s not a great long-term solution, but until hosts support TLS there isn’t much else to do (that I know of – leave a comment if you know better). In the meantime, here’s a Filezilla forum thread on the issue that affected users can follow.

Hope this helps someone!

How to globally change the font size in IntelliJ IDEA

Today I learned… that you can change the font size in IntelliJ IDEA across all editor tabs and consistently! It’s not a missing feature and you don’t have to rely on the awkward trackpad zoom feature.

Here’s how you do it (on a Mac, anyway).

Go to IntelliJ IDEA —> Preferences —> Editor —> Colors & Fonts —> Font

Screen Shot 2015-02-19 at 9.05.28 AM

On the right, look for Scheme name.

Screen Shot 2015-02-19 at 9.04.34 AM

Click the Save As… button and give it a unique name.

Screen Shot 2015-02-19 at 9.03.47 AM

This step is essential! If you don’t do this step, changes you make to Size further down won’t stick, and IntelliJ won’t tell you why. (The popup is titled “Save Color Scheme” so it’s easy to see why users might overlook its effect on the ability to save font size.)

Now alter the Size setting and you should see the typeface preview change to reflect your input. Press OK and you’re done!

Screen Shot 2015-02-19 at 9.06.31 AM

Googling this problem, I found some conflicting answers about how to increase IntelliJ’s font size.

Many IntelliJ users recommend the “trackpad zoom” solution (which you have to enable in Preferences —> General —> Change font size (Zoom) with Command+ Mouse Wheel), but relying on zooming in results in a patchwork of mismatched zoom levels across all your tabs, and you have to manually re-zoom each tab. I wanted a consistently larger font-size across all tabs, with no manual zoom step, and this did the trick!

Git: How to automatically add the branch name to the end of every commit message

Today I learned… that git can be customized in a number of ways by using its hooks!

On my engineering team, it’s our convention to name feature branches after their corresponding JIRA issues. Likewise, we include the branch (issue) name in every commit message.

$ git commit -m "Fixed the thingy jira/story/ourproject-4555"

Including the branch name / issue name like this means the commit will show up in JIRA linked from the issue itself! Pretty cool. I’ve worked with JIRA a lot over the years, but I’ve never been on a team that actually integrates it with git like this, and the organizational benefits are worth the extra effort.

Alas, it is all too easy to forget to add the branch name at the end of a commit, and I got tired of amending my git messages (which is easy enough: git commit –amend will do it). I knew there had to be a way to automate this.

Adding a git hook

  1. Go into your project’s repo
  2. Open ./git/hooks directory (remember, .git is a hidden directory by default)
  3. Make a copy of .git/hooks/prepare-commit-msg.sample, paste it into the same folder, and remove the .sample extension. You should have a file named simply prepare-commit-msg with no extension.
  4. Paste this into the new file:
# Automatically adds branch name to the end of every commit message.

NAME=$(git branch | grep '*' | sed 's/* //')
echo $(cat "$1") "$NAME" > "$1"

The first line gets the git branch name and sets it to “NAME”.

The second line echoes the contents of your commit message (represented as $1), adds “NAME” to it, and replaces $1 with your message + NAME.

Just save the file and try it out in bash now! No need to reload bash or source anything.

Here’s the gist

# Automatically adds branch name to the end of every commit message. Handy for git/JIRA integrations.

# Make a copy of .git/hooks/prepare-commit-msg.sample in your repo and name the file prepare-commit-msg (remove "sample" extension)
# Paste the following into the file, save the file, and try it out in bash

NAME=$(git branch | grep '*' | sed 's/* //') 
echo $(cat "$1") "$NAME" > "$1"

More examples

There are many ways to approach this problem, and you can do a lot of sophisticated things with Git hooks. Here are some resources I found helpful in learning about git hooks and git in general:

Gulp – Build Automation for Javascript

Tired of restarting your server manually, over and over again, whenever you change something? Straining under the labor of recompiling your Less / Sass dozens of times an hour? No more. Let a robot do it for you. A robot named Gulp.

Why Gulp?

First off, I should mention that there are several build automation tools written specifically for Node.js apps. Grunt is probably the most popular, but there’s also Jake and a couple others. I started with Grunt, but I’m liking Gulp more and more as I use it. It’s faster, more flexible, and just “feels” more intuitive and straightforward.

basedHonestly, though, which one you use isn’t important. All that’s important is that you use SOME KIND of build automator / task runner. Over the long run, it will save you hours of repetitive, frustrating, mindless drudgery. Sound good? Read on.

Installing Gulp

To use Gulp in your app, you must install it globally onto your system, as well as locally in your project directory. Use the following commands in terminal, while inside your project dir:

npm install --global gulp
npm install --save-dev gulp
touch gulpfile.js

Your First Gulp Task

Open the gulpfile.js you just made in a text editor and put in the following:
var gulp = require('gulp');
gulp.task('default', function() {
  console.log('If you can read this, gulp is working!');
});

Go back into your terminal, type gulp, and press Enter. You should see the following output:

[20:33:59] Using gulpfile ~/YourProjectFolderHere/gulpfile.js
[20:33:59] Starting 'default'...
If you can read this, gulp is working!
[20:33:59] Finished 'default' after 74 μs

So what happened here? We created a task called default in the gulpfile, which calls a function when the task is run. That function then performs a console log. Using the gulp command with nothing after it will run any task named default. It’s a good idea to always have a default task that does the important work of building and running your project. That way, anybody else who has to work with your project can just type gulp in the terminal and see it run without having to paw through your code.

Your First Useful Gulp Task

So that was fun, but not particularly worthwhile. Let’s do something useful with Gulp!

Let’s say you have a directory full of JavaScript files, a directory called ‘js’. All these files need to be copied over to a directory called ‘build’ before you can publish your app. No problem! Put this into your gulpfile:

var gulp = require('gulp');

gulp.task('copy-js', function () {
  gulp.src(['js/**/*.js'])
    .pipe(gulp.dest('build'));
});

gulp.task('default', ['copy-js']);

There’s a lot going on here, so I’ll explain bit-by-bit:

  • We created a new task called copy-js, which will do all our copying for us.
  • The first line inside that task, beginning with gulp.src, tells gulp where to look for the files we want to copy. That bunch of /s and *s we gave it is a pattern-matching string called a glob. Here’s how to interpret this glob:
    • The js/ part tells gulp to look inside the directory named ‘js’.
    • The **/ part tells gulp to look inside any subdirectories within the ‘js’ directory.
    • The *.js part tells gulp to find all files that end with the .js file extension.
  • On the next line, we chain a method onto the end of gulp.src… specifically, the .pipe() method. .pipe() takes the output of the previous method (i.e., the .js files we found) and lets us use it as input for another method, just like a unix pipe. This is extremely useful, as you’ll soon see.
  • .pipe() passes the files we found to gulp.dest(‘build’)gulp.dest() is used to save files to a particular location. Which location? Why, the one we told it: the ‘build’ directory.
  • Finally (and importantly!) we changed the default task. Instead of executing a function, default will now execute a list of sub-tasks. For now, we just want it to execute our copy-js task.

Now, if you type gulp into the terminal, any JavaScript files in the ‘js’ directory will be copied into the ‘build’ directory. Gulp will even create a ‘build’ directory for you if it can’t find one. How thoughtful!

Watch This

“This is all well and good,” you might be thinking, “but how does this actually save me time?” After all, you still have to keep typing gulp into the terminal every time you want this copy and paste to happen, right?

The answer is no, you don’t. Gulp can run tasks for you, automatically. Enter gulp.watch():

var gulp = require('gulp');

var jsDir = 'js/**/*.js';

gulp.task('copy-js', function () {
  gulp.src([jsDir])
    .pipe(gulp.dest('build'));
});

gulp.task('watch-js', function () {
  gulp.watch(jsDir, ['copy-js'])
    .on('change', function (event) {
      console.log('File ' + event.path + ' was ' + event.type);
    });
});

gulp.task('default', ['watch-js']);

Ok, so what happened here?

  • We made a new task called watch-js. When this task is executed, gulp.watch() will keep a close eye on the directory we tell it, watching for files inside to change. When they do, the tasks in the array we provide will be executed… in this case, the copy-js task.
    • To put it simply, whenever we change a .js file, it’ll be copied over automatically. How cool is that?
  • We chained .on() to the end of gulp.watch(). This lets us execute code when certain conditions are met. In this case, when a file changes, we execute a function. This function uses the event parameter to let us console log which file changed, and how it was changed (added, changed, deleted, etc.)
  • Also, we put the JavaScript directory glob into a separate var called jsDir, which we use in both the copy-js and watch-js task. That way, we can make sure it stays consistent.
  •  Finally, we change the default task to execute watch-js when it’s called. By the way, you’ll notice this is an array; we can comma-separate multiple sub-task names to be called here, if we choose.

Sweet! What Else?

Gulp can help you automate all kinds of development-related tasks, including but not limited to:

  • Linting
  • Unit / Integration Testing
  • Bundling / Concatenation
  • Minifying / Compression
  • CSS pre-processor compilation (i.e. Sass / Less)
  • Image resizing / processing
  • Asset versioning
  • Running shell commands

To learn more, check out Gulp’s documentation and browse their extensive, searchable list of plugins. To use a plugin, npm install it, require it at the top of your gulpfile as a variable, and then use it based on the plugin’s documentation. Like the following example does with gulp-sass:

var gulp = require('gulp');
var sass = require('gulp-sass');

gulp.task('default', function() {
  gulp.src('sass/*.scss')
    .pipe(sass())
    .pipe(gulp.dest('css'));
});

That should be enough to get you started. Happy gulping!

Install NPM packages globally without sudo

To be an effective full-stack Javascript developer, you pretty much have to use npm (Node Packaged Modules)Npm-logo.svg

npm is the official package manager for Node.js. Using it allows you to easily install packages such as underscore, express, grunt, gulp, socket.io from the command line. (read more about npm on Wikipedia and npmjs.org).

Many of these packages need to be installed globally to be used, like so:

$ npm install -g grunt-cli

The -g denotes a global install.

But, in some environments, attempting to install globally without including sudo might give you this sort of error message:

npm ERR! Error: EACCES, open '/Users/YourNameHere/.npm/-/all/.cache.json'
 npm ERR! { [Error: EACCES, open '/Users/YourNameHere/.npm/-/all/.cache.json']
 npm ERR! errno: 3,
 npm ERR! code: 'EACCES',
 npm ERR! path: '/Users/YourNameHere/.npm/-/all/.cache.json' }
 npm ERR!
 npm ERR! Please try running this command again as root/Administrator.

Should you just do what the error message says and run this command as root using sudo? The short answer is: NO! Packages can run their own scripts, which makes installing them with sudo about as safe as shaving with a blowtorch.

There are many ways to solve this problem, but the easiest I’ve found is to simply change what folder npm installs packages into. Just use this terminal command:

npm config set prefix ~/npm

This tells npm to install packages into your home directory in an “npm” subfolder. It will automatically create this new subfolder the first time you globally install a package.

But we’re not done yet! You also need to modify your $PATH to include this new folder. To do this, open the config file of your shell in your favorite text editor. This is most likely “.bashrc” in your home directory, or “.zshrc” if you use Z shell. Once you have the file open, search for a line that looks like this:

export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"

At the end of this line, add:

 ":$HOME/npm/bin".

After I made the change, my path line looked like this:

export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:$HOME/npm/bin"

The $HOME part of that just tells the shell to look in your home directory for a folder called “npm”, and then a folder inside that called “bin”. Once you’ve done these two things, you should now be able to install packages globally in npm without using sudo. Enjoy the tremendous feeling of freedom this brings.

Automate Express server restarts with Nodemon

Today I learned… that you don’t have to navigate to Terminal, press CTRL + C to stop the server, and re-enter node server.js to start up the express server again in order to see your latest work in the browser.

There’s a better way: get Nodemon and automate this repetitive process!

Nodemon will watch the files in the directory you run it in and restart your server automatically when it detects a change.

To install Nodemon:

$ npm -g install nodemon

To get Nodemon running:

$ nodemon server.js

Leave that Terminal window open and let Nodemon work its magic.