Everything is amazing, and nobody's happy. — Louis CK

Amazon S3 experienced an outage yesterday, making some SaaS products appear broken while others downright unusable. This outage took down large parts of the internet. Ouch.

The outage wasn't limited to online. Some Internet of Things gadgets depend on services like IFTTT to integrate between the gadget interface in the real world and the Internet-connected world. It feels a little like magic when it works as planned.

But when that magic is broken?

Mmm. Can't turn some of my lights on at home cos @IFTTT is down. Welcome to the future!@internetofshit pic.twitter.com/CPw0rGaKR5

— Stuart Thomas (@stuartthomas) February 28, 2017

Not being able to turn on the lights is one thing. What if an outage becomes a fire hazard?

Not to raise alarm here or anything, but I'm unable to turn off my #IoT oven ever since #s3 went down... It's getting kinda toasty in here

— pronto@ °.• 🕚 (@_pronto_) February 28, 2017

During the outage the Amazon Web Services status page, which itself reports on service availability, ironically experienced an outage itself, due to its own reliance on S3.

The dashboard not changing color is related to S3 issue. See the banner at the top of the dashboard for updates.

— Amazon Web Services (@awscloud) February 28, 2017

When they breakdown, the technologies that empowered us so effortlessly now limit us. We become demoted from value-add to net negative. And it's all beyond our control because we don't understand the magic behind that technology's curtain. We look to project failure onto someone else--onto the real reason we can't do more.

Who is to blame?

When the Magic Runs Out

Grant loves Twitter. It's a quick escape on his 30-minute lunch break that's all too short before he crawls back to his call-center cubicle to proposition another unsuspecting victim.

Finally free from the clutches of his headset, Grant fires up his favorite Twitter app, TweetPad. He just wants a laugh, one laugh, to calm the defeat he feels in his heart from making failed direct sales calls all day. But today the funny images aren't showing up. Tapping those familiar buttons does nothing. Pieces of the interface are missing. TweetPad is broken. This is bullshit.

Grant blasts TweetPad in his next tweet, posted from another Twitter client that doesn't rely on S3. TweetPad loses trust amongst Grant's followers as a reliable service. In turn, attracting less users, resulting in less growth, and less profits.

Who is to blame? TweetPad engineers, for relying on a single point of failure? Amazon, for not providing a reliable service? An Amazon engineering team, who didn't design a fault-tolerant system?

The Great Lawrence Internet Outage of January 2017

Where I live in Lawrence, Kansas, a college town of about 90,000, a single ISP supplies internet to the majority of residential and business customers. One afternoon, the #LFK hashtag was ablaze with reports the internet was down. The ISP, fittingly named "WOW!", (yes, that's actually their name) became a parody of itself.

Some local businesses even tweeted they were unable to accept business because their point-of-sale terminals were unavailable without an internet connection.

Who is to blame? The business, for not having a backup payment processor? The ISP, for not providing a reliable internet service? The point-of-sale terminal, for designing a service with a single point of failure?

(According to the ISP, a fiber-optic cable was severed due to a car crash near a major junction box. Who is to blame for that one?)

The Chain of Blame

Where does the chain of blame end when everything we do is dependent on so many lower-level things going just right?

Ben and Katie's favorite restaurant is closed because the kitchen is unable to prepare food without electricity. Customers are understandably frustrated, but Ben and Katie feel their anniversary dinner plans are ruined. The businesses lose real profits and the unconscious trust they had in it.

Who is to blame? The restaurant, for not having a redundant power supply? The electric company, for not providing reliable service? The electrician, who improperly installed the down-converter that blew up causing the outage?

The driver's side tire of a late-model four-door car flies off while zipping down the highway. This freak accident causes a crash, claiming the life of the car's driver. Twenty-eight year old Saura was hurrying back to her laboratory to analyze the latest samples in her groundbreaking biology research. Sadly, the world may never know the outcome.

Who is to blame? The mechanic, for not properly re-tightening the lug nuts? Saura, for not checking her car was in proper working condition before driving? The car manufacturer for designing a non-standard wheel system that cut costs to meet shareholder demands for profits?

Software Ownership

Every technology is powered by a magic that we take for granted. Is there magic in software development?

Developers routinely pull open-source libraries into their projects. If a developer can re-use code someone else has already written, it leaves them to focus on higher-level problems and deliver more value more quickly, yada yada. Congrats, that's why we're all here, to build "technology."

However nearly all developers think nothing of how that code came to be or what quality it bares beyond the magic curtain of its public interface.

After a few weeks, a problem crops up. The developer narrows down the problem to a bug in the open-source library that miscalculates payments. Customers are being charged twice the normal price. Oops.

The negative app store reviews practically write themselves. In the short term, no amount of damage control apologies and free coupons will repair the lost trust between the company and its paying customers.

Who is to blame? The developer who installed the code, for not reviewing the code themselves? The original developer who wrote the bug in the open-source library? What about the open-source community itself, for not seeing and fixing the bug sooner? (Hint: Almost all open-source libraries contain licenses releasing their authors from liability. Good luck filing a lawsuit.)

Stop Blame Behind the Magic Curtain

When a failure occurs within your team or company, you know all the details. It wasn't personally your fault and maybe not even your team's fault.

Searching for blame and fault behind the curtain is toxic. It creates a Blame Culture that drags down the team's creativity and ability to work together to do amazing work. Even searching for blame erodes trust among the very people who should trust each other the most in your organization.

Instead, when something goes wrong, collectively accept the blame. "Yep. It was ME. It was US. We did it. And we're gonna to fix it."

When bugs reach customers it's not likely because the author was incompetent. Instead maybe the team's code review practice was just not good enough. Maybe their checklists weren't followed or simply didn't exist.

Throw a party! A weakness has been discovered! This otherwise bad situation becomes an opportunity to improve.

Task those who could be blamed with contributing to a plan that keeps it from happening again.

Stop Blame Outside the Magic Curtain

We as makers especially fall victim to blaming technology and its makers when their magic runs out. Certainly we would have never made such a stupid mistake. I mean, how can you screw up Copy & Paste?!

Have compassion. Cultivate empathy. Everyone else is working just as hard to make great things and empower you and everyone else to build even bigger things.

This goes for everything we use in life. The person, people and team who made that tool are probably working their hardest to make the best thing they can given their constraints on time, talent and budget.

Stop for a moment and forgive their oversight. Think of the power you gain from when their tool works correctly. It's much, much more than the small crash or deficiency you just found.

If you can help, submit a bug ticket. Or even submit that diagnostic report after the app launches again. And don't include some smart ass insult.

Stop perpetuate blame culture on both sides of the curtain.

That includes not trashing Amazon for the S3 outage yesterday, and all the products relying on AWS to build great things.

I mean, give it a second. It's going to space!

I took this photo from a rooftop terrace where I digested what happened just an hour after the event.

January 26, 2017 – Playa Bejuco, Costa Rica

The sun is plunging into the ocean. This might be the most beautiful sunset I've ever seen. Because an hour ago that was me, awash in the ocean, in danger of going under.

I almost died today.

I'm fortunate to tell this story instead of being its subject. What happened?

I wrap work for the day at 3pm, still high on shipping a new software feature for my work project. How do I spend the last 3 hours of sunlight while staying 250 meters from the beach?

Mike is deep in work but I can't contain my excitement. "Yo… wanna go surfing?"

He hasn't surfed before but after our group's successful surf lessons a few days ago I'm practically a master, right?

Inspecting the two new communal surfboards at our condo, one of the leashes has snapped. That's cool, we can share.

We hit the beach and I do my best Argentinian surf teacher impression, recounting the basics she taught the group: center line, standing up, and holding a martial arts stance for balance. Simple.

The high crashing waves in the distance only add to my excitement. I tighten the velcro-strap surfboard leash to my right ankle and lead us high stepping into the water.

The low tide creates a strong pull that almost sweeps us off our feet. I remain focused on reciting to Mike the tips our instructor gave us.

In position face-down on the surfboard looking over my shoulder, I try timing a few waves. The slick board is difficult to stand up on. Maybe we didn't put enough wax on it? No worries, I'm just warming up.

Meanwhile, Mike idly steadies himself against the unrelenting waves, observing and taking mental notes. He jumps, bobbing up and down to keep his head above each rolling wave before landing again in water cresting at his chest.

Glaring at the shoreline I realize the sun loungers look more like colorful gummy bears than people. We're pretty far out, huh? How did we get here?

The strong undercurrent has dragged us out a meter or two with each wave. And Mike hasn't even had a chance yet. OK, let's get back to shallow water so he can give it a try.

Swimming towards the shore for several minutes the gummy bears continue shrinking. Is this the twilight zone? "This isn't fair," I think to myself. It's not supposed to work like this. I'm a strong swimmer and can't make headway. Mike trails behind.

I have a bright idea: let's both hop on the board and kick our feet to shore. The next wave comes quickly and the board jets out of control towards shore but without us on it. The rubber tubing of the leash tugs tightly at my ankle.

Mike pogo-sticks on one foot, his mouth slipping below the previously chest-high water. Yo, we're way deep, let's get a move on!

The leash ties itself into tangles as we both try jumping on again. Of course, the leash! I rip off the velcro strap thinking the board will be easier to maneuver.

The next killer wave rips the board toward the shore, forcing me under as it crashes, plunging the board into the white crest of the waves.

I exhale loudly as my head resurfaces looking towards the shore for Mike and the board. I feel naked without the leash tugging at my ankle. Surely Mike is still holding on…

Mike is a meter in front of me. "Where's the board?!" I exclaim. "It's gone…"

Early stages of panic start to set in except there is no music like in the movies. This is real life. This is real and this is really happening. We have to get back to shore.

The harder we paddle the harder the waves hit.

Within a few minutes Mike starts losing composure. Lagging behind me he is tiring quickly. His face familiar, like every scary movie where a character is sucked into the jaws of a snarling blue monster.

I give up the ground I have gained to grab his hand. I got us into this mess and I'm going to get us out.

Suddenly I'm responsible for two lives. Trying to swim sideways pulling for both of us I too start to fatigue. The sand gushes up between our toes as each wave reclaims any progress our arms make between waves.

I quickly realize we aren't going to make it back by ourselves. I start hollering for help. Mike echoes me.

A body boarder 100 meters away makes eye contact with us, and seeing how far out we are, waves his arm signaling us to move back to shore. But with the board gone we have nothing to keep us afloat.

Ghastly scenarios begin running through my mind. Not life flashing before my eyes, but the impending worst case scenario. My hometown news recounting my death between Donald Trump outrage and a weekend snow forecast.

I envision my family, coworkers, the woman I'm dating, friends, fellow travelers, all trying to piece together our last whereabouts. The few people that know we went surfing recounting the time they last saw us. Their descriptions of how excitedly we bounded to the beach. Maybe they would even remember us fondly…

I snap out of it. Back to Mike. He's 30 seconds from full on panic, his eyes darting everywhere looking for certainty. Another worst case scenario appears: Mike dipping below the surface and not coming back up. Could I swim after him and save us both?

"Amigo! Amigo! Amigo!" we holler to the body boarder. It's no use. Our voices are swallowed by the same roaring waves chomping at our heels.

Our hands remain clasped. Their tight grip says, "Stay with me," and "I'm not leaving you." We're each left with one hand switching between treading water and waving for help, as if we're not sure which one is more likely to save us.

I'm going to die out here.

What feels like a hundred waves later, two men appear wading towards us. An older man has our surfboard that must have washed ashore. He is flanked by a shirt-less bronzed man, his long sun-bleached gray hair and thick dad-like handlebar mustache somehow signal he is calm and collected in this otherwise emergency situation.

"What the fuck are you guys doin'?!?!" His English is impeccable.

"We're fuckin' amateurs…!" I exclaim. He barks at us to grab onto the board and ride a wave back in. It sounds so simple, now.

The same waves punishing us a moment earlier are now our golden parachute to safety. We thrust toward shore on a wave that rolls all the way into shallow water. The surf angels that lifted our travel-mates into standing position during surfing lessons earlier this week are working overtime.

The sand becomes more sure-footed with each step. The waves less strong. Baby waves splash our ankles.

We share a glance. The silence between us says more than words could.

"I'm really sorry, Mike," I say breaking the sound barrier.

"It's all good, dude…" he says, way less casually than such words normally warrant.

Walking back to shore our tails remain firmly tucked between our legs. Judgment quickly follows us from behind.

"You guys are lucky…" The bronzed man's words trail off over a cliff. "We pull a lot of dead bodies out of here." Our eyes lower.

"Thank you…" He's heard my sheepish appreciation many times before today.

"How much longer you guys down here? We teach rip tide courses." He gestures toward a lookout tower up the coast. "It's how the volunteer life guards make their money." Somehow the $40 fee pales compared to saving our lives.

The setting sun shines brightly behind him, outlining his towering stance in a golden halo as we sink against the driftwood log peering up at him.

"My name's Bob." Is there a Saint Bob? There is now.

Bob leaves and certifies our reckless optimism. Mike and I hold silent a few more minutes to catch our breath. I apologize several more times.

"Welp… this will make a great story at Shot, Show and Tell later tonight…"

Lessons Learned

• Learn to identify rip currents and undercurrents before going to the beach. Ask the locals when you arrive.
• Every surfer should have their own board.
• Never unhooked the leash from your leg.
• If caught in a rip current or strong undercurrent, do not swim directly back to shore. Swim sideways, parallel to shore, then once you no longer feel the strong pull, swim diagonally back to shore.
• If you are inexperienced, admit it to yourself, and do not put another person's life at risk based on your overconfidence.

I'm a little old school. In an age of purchasing tracks I still prefer listening to albums start to finish. Appreciating how tracks are arranged one after another and how they transition from one to the next is very important.

Music and coffee are my lifeblood. Coffee is difficult to share, so please accept my favorite albums released in 2016, including statistics on all music fed into my ears.

The below list is unranked and in alphabetical order.

The Black Queen - Fever Daydream

If Trent Reznor never made music again (he snuck in a new Nine Inch Nails EP in 2016), The Black Queen could pick up the industrial electronic torch from where Nine Inch Nails leaves off.

While not directly affiliation with Reznor, The Black Queen features two former NIN alumni who can't hide their successful past by grinding out some of the best industrial-rock music programming since The Fragile. Frontman and vocalist Greg Puciato (The Dillinger Escape Plan) presents his softer side with subdued effect-laden vocals on top of careening 1980's synth-pop reincarnate.

Wait, what year is it?!

Cubicolor - Brainsugar

The cooing vocals of Peter Kriek on the Cubicolor debut album Brainsugar echo those of Thom York (Radiohead), while its entrancing lo-fi beats spin a subzero cocoon as if Boards of Canada were remixed by a deep house DJ in a cold Montreal basement.

Signed to label Anjunadeep created by electronic masterminds Above & Beyond, Brainsugar resonates wet atmospheric synths under minimalistic guitar and optimistic electronic beats.

Die hard Radiohead fans will scoff Brainsugar a poor man's interlude, wondering no-man's land somewhere between Kid A and Amnesiac. Met with an open mind Cubicolor carve their own niche into a crowd of bedroom-borne imitators.

The Dillinger Escape Plan - Dissociation

The Dillinger Escape Plan are an acquired taste. As in, they might last 5 seconds in Dad's stereo before being flung out the window leaving you both grounded and without a Dillinger album.

The band's final studio album Dissociation pushes all of their genre-bending extremes to their conclusion. Playing to their strengths, the band waver between unchecked hostility and dizzying shoe gaze, quite often within the same song. Haunts of Aphex Twin are scattered across several tracks (the song Fugue sounding directly lifted from Drukqs) as if we accidentally entered shuffle mode.

Car crash anthems like the track Honeysuckle dip in and out of jazz-inspired lulls, early Norma Jean-esque guitar-screeching interludes, crushing blast beats and quite literally some of the most brutal metal breakdowns ever recorded.

The line between incoherent rambling and genius is thin. Dissociation firmly lands in the latter camp and serves a fitting curtain call for a genre-defining band.

Every Time I Die - Low Teens

Seven albums deep Every Time I Die continue bravely bending the hardcore metal genre into places others can't touch. Keith Buckley's vivid lyrics and scathing analogies breath fire in as many words.

Nostalgic nods hearken to their hardcore-punk influence, complete with fist-pumping chant-alongs littered among crunching southern rock riffs. Each track bravely rolls the dice between genre, speed and tone. While not always successful the band will never fall victim to being labeled stale.

The secret to ETID's continually evolving sound might be regularly employing new drummers. Low Teens is the first offering featuring the band's third drummer, Daniel Davison (Underoath, Norma Jean), whose technical fills, off-time yet in-time hits and punk vibe play perfect compliments to the band's dueling guitars.

Low Teens is a batch of fresh jams that won't win any new fans but will infect their faithful with lyrical ear worms such as (Stare at the sun / Stare directly at the sun) and (I timely ripped into this world / I was born again as a girl) and the boastful lines (I'm joining a cult / joining a cult / joining a cult).

Metallica - Hardwired… To Self-Destruct

Metallica have settled their debts and risen from the ashes of St. Anger's snare drum to release the year's biggest surprise: a far from terrible Metallica album. In fact, this is Metallica at their best in 20 years.

Read my full review of Hardwired… To Self-Destruct.

Moon Tooth - Chromaparagon

The debut album from Moon Tooth forays into the crowded progressive metal genre and emerges with an original sound despite blending styles of their predecessors.

Heavy influence from early nu-metal titans System of a Down and oft-forgotten The Apex Theory are plentiful throughout the album. Intricate finger-style guitar work plays perfectly against progressive fill-oriented drums (think a less-technical Between The Buried and Me) while bass lines wonder their own path up and down the scale behind a wide-range of vocal ranges and styles.

The band are incredibly talented musicians and take turns showcasing their prowess. With so much musicality packed into each track no two listens yield the same listen. Repeat listens are rewarded with the satisfaction of having memorized many of the intricate musical parts, clearing way to discover nuance that was overshadowed on prior listens.

Don't be surprised to see these guys opening for the genre's more seasoned acts in 2017 as they make a well-deserved name for themselves.

2016 Music By The Numbers

Thanks to Last.fm for tracking my every listen since June 2005.

Total Tracks: 4,487 different tracks
Total Albums: 1,385 different albums
Total Artists: 777 different artists

Top Track: The Black Queen - Ice to Never (61 listens)
Top Album: Every Time I Die - Low Teens (527 listens)
Top Artist: Metallica (897 listens)

Top Listens by Album

Top Listens by Artist

Listen on Spotify while reading.

Look. Hardwired… To Self-Destruct is to Metallica as The Force Awakens is to Star Wars. There is no way the "new" can ever top the "old." The best "new" can only ever co-exist with the "old" by simply not fucking up its reputation. With me so far?

Metallica endured a rough ride releasing Load (1996), Reload (1997) and Garage Inc. (1998) mostly departing from the shredding thrash metal songs that made them kings of the genre in the 80's. Long-time fans took it personal, heralding the band sell-outs.

Not content with a musical fall from grace the band confirmed their critics by suing Napster in 2000, drawing a fireball of criticism from progressive technoliterate music fans, only confirming Metallica as washed up rockers playing right into the greedy, technophobic stereotype.

Then what can only be rationalized as retaliation, the band released St. Anger (2003) and Death Magnetic (2008), both plagued by poor sound production choices and downright terrible songwriting. Undoubtedly these were two of the worst albums ever released by a legitimate metal band.

Frankly everyone (including me) relished watching the wreckage burn.

But somehow with Hardwired… Metallica have agreed to settle debts with their fanbase and risen like a phoenix from the ashes of St. Anger's snare drum. Forgiving the derpy album artwork, most tracks drip of nostalgic as if reincarnated B-sides haunting the band's golden years.

Opening track Hardwired is the shortest on the album at 3:11 but rips straight for the jugular landing more unrelenting jabs and body shots than any song twice its length.

The swelling intro and soaring riffs on Moth Into Flame feature the album's catchiest lyrical hooks (Same rise and fall / Who cares at all / Seduced by fame / A moth into the flame) all together masquerading as a really damn good Seek and Destroy rip off.

But the album begins coasting downhill. The midpoint feels lonely and rhythmically repetitive by Halo On Fire, which rolls into Confusion, then saved by some solid southern rock riffs on ManUNkind, and back to the lonely road with Here Comes Revenge. After numerous listens these tracks fade into the background, only briefly cropping into the conscious with catchy ear worm chorus lyrics.

However the final track Spit Out The Bone accelerates out of cruise control with a crisp, almost drum-roll snare drum before an upbeat chugga-chugga-chugga-chugga riff reminds us Metallica have life left in them. Numerous breakdowns, wailing guitar solos and double-bass round off the album in the best way possible: leaving us hungry for another listen. The "Repeat Album" setting feeds that hunger ripping straight back into the opening track Hardwired.

The album's Deluxe Edition track Lords of Summer says it best: (It's been a long lonely winter / The lords of summer have returned!)

Let's be real: only Metallica could save Metallica. With Hardwired… To Self-Destruct the band have redeemed over 20 years of disappointing music. Not with their best album of all time, or even of that 20 year span, but one sure to reclaim displaced fans.

While far from flawless or without fat fit for trimming, Hardwired… is a return to greatness by simply not fucking it up.

Bonus Content

The band's YouTube channel is straight killing it releasing new content almost daily. For Hardwired… the band shot music videos for all thirteen tracks.

Each video is stylistically unique, ranging from fully animated story-arc narratives, to making-of in-the-studio style overlaid with album tracks, to quintessentially playing in a warehouse, to professionally recorded concert performances. Those appreciating the lost art of the music video will find value in watching each one.

1. Hardwired
2. Atlas, Rise!
3. Now That We’re Dead
4. Moth Into Flame (best song on the album?)
5. Dream No More
6. Halo on Fire
7. Confusion
8. ManUNkind (best video?)
9. Here Comes Revenge
10. Am I Savage?
11. Murder One
12. Spit Out the Bone
13. Lords of Summer

Yes, that is Metallica, posting their entire new album on YouTube. For free.

My, my, how times have changed.

Last week I delivered a talk Rethinking Loops at DrupalCon 2016 in New Orleans. DrupalCon is a HUGE conference, with over 3000+ attendees. Easily the largest conference I have attended, let alone spoken at.

The conference records all sessions and posts them within hours of the talk. Very impressive turnaround time!

My session is below and slides are available: Rethinking Loops slides

Rethinking Loops

Being the last session, on the last day, where attendees also had the option of attending a session by the creator of Drupal, I was a little worried if people would even show up…

I walked in fifteen minutes early and the room was already 75% full, with more filing in after me. Relief. I've heard other speakers grow more anxious the more people are in the room. Not me. I feed off of it. Especially a small, intimate, packed room.

A packed room instantly validates what I'm speaking about and gives me confidence that what I have to say is important, insightful and valuable—a feeling I've struggled with most of my life.

Attendee Feedback

DrupalCon facilitates speaker feedback thru their website. I always appreciate feedback so I can improve my talk and overall become a better speaker. If you have any feedback after viewing the talk above please ping me.

Attendees who were gracious enough to submit feedback all gave very positive remarks. Some of the highlights:

One of the very best sessions at DrupalCon this year, and one of the best speakers I've heard. I didn't completely agree with the speaker, but I love seeing different ways of thinking.

Overall excellent presentation. Probably the best one I saw (I only missed 3 slots)

One of the most valuable sessions I attended. Can't wait to pass this on to my colleagues at our next code review.

Very smooth presentation. Even at 2pm on the last day of conf. Had me wide awake and interested. I wish more sessions were like this.

Attendees and other speakers have specifically raved about how my talk uses pre-recorded video to demonstrate refactoring code in one segment. It's actually ten separate videos across ten separate slides, but with a few slideshow and production tricks it appears seamless when talking over it. Here's the exact point in the recorded session where I show this. Maybe I'll do a future post on how to produce it.

Social: First-Time Attendee Social

Because of its scale the conference does well to acclimate first-time attendees with its First-Time Attendee Social the night before the first day of sessions.

We were handed Bingo cards where each cell described a person we needed to meet. Things like "From the East coast" and "Knows what a BOF is" and "Member of the Drupal Association." We were ushered into an intentionally small area, given free beer, water and cookies, and bounced around like pinballs meeting each other and morphing between conversational groups. The format worked well and I talked with several people again throughout the conference.

Coincidentally I met a person who lives in the same city as me (Lawrence, Kansas, population ~90,000) that's involved with publishing Drupal Watchdog, a magazine with articles interesting to Drupal users and developers. Funny we had to travel all the way to New Orleans to meet each other.

Social: Pub Crawl

One of the attendees took it upon himself to loosely organize a pub crawl after the First-Time Attendee Social. I like beer, and meeting other developers over beer, so I knew before the event I'd be going.

Three of us met in the lobby to walk to the first stop, The Courtyard Brewery, about a half mile away. We arrived and quickly saw groups of other conference attendees scattered about. Many attendees I talked with worked in higher education, mostly state universities, while others worked for agencies with quick turn around times. The groups intermingled for an hour before heading to the second bar, The Avenue.

The group eventually ended up out back on the patio, and after meeting a few other people that work with remote development teams overseas I had to call it a night about 10:00pm. I still had my backpack/laptop with me and I didn't want it to get too late, knowing I had a half-mile walk between my cable car stop and where I was staying. Walking by myself I simply didn't want to be a target for theft. I hoped the cable car right outside the bar on St. Charles street and made it home without incident.

Event: Pre-Note

Tuesday, the first day of conference sessions, began with what can best be described as musical theater. The "pre-note" was endlessly hyped at the opening ceremony and first-time attendee social, so I was curious to see what it was all about. At 8:00am I felt I sacrificed a little sleep to get there to see it and wanted it to be worth my time.

My skepticism peaked as the event began with some awkward crowd-rousing "I can't hear you! Are you excited?!" type banter from the first people on stage. But by the end I realized it was absolutely like NOTHING I had ever seen at a development conference and that many other tech communities could learn a lot from this community.

Several prominent contributors to the Drupal community sporting intricate headdresses and shiny skin-tight full-body leotards took the stage, their costumes leaving nothing to the imagination. The actors' lines and musical performances were delivered with as much gracefulness as a high school drama, while their individual singing voices were surprisingly good—an obvious reason they were selected to lead the performance.

As the performance wore on and the other casts of characters were introduced and performed their dance and musical numbers, all with equally cheesy lyrics and all spoofing classic pop culture songs, I realized something…

This performance wasn't trying to be anything. Instead it was the coming together of the community's most prominent figureheads and splaying themselves in a highly vulnerable and slightly embarrassing light. All for the sake of making us smile, laugh, and root for them. It reinforced how everyone attending, whether speaker, volunteer, core committer, or simple attendee, were all simply people. A reminder that those people are not their titles within the community, nor their contributions to the ecosystem. They are people just like you and me.

The pre-note was a microcosm of the Drupal community: people from widely varying backgrounds willing to accept and embrace anyone wishing to take part. And most importantly a community that doesn't take themselves too seriously. It's truly rare to see all of these characteristics within one community set within an industry conspiring in favor of stereotypical ultra-serious, ultra-assertive, yet often ultra-shy geeks.

I must give special mention to Larry Garfield's solo dancing and singing number. I've met and talked with Larry several times at previous conferences, and before the pre-note I would've described him as a vocal but reserved person. Maybe he's a lot like you. But after seeing him get up on stage wearing only pants, his signature black vest yet no blue undershirt, then singing and dancing all over the stage to the tune of "I'm Too Sexy," I have an even higher esteemed respect for everything Larry does for the wider PHP and Drupal communities. There's simply nothing he can't and won't do.

Here's the video stream beginning at Larry's performance. He's absolutely a much better singer and dancer than me. Thank you for everything you do, Larry.

Sessions

With three days packed full of sessions it was sometimes hard to decide which talks to attend. Two sessions I attended stood out in their relevancy to modern web application development:

GraphQL Meets Drupal

GraphQL is a game-changer in querying for data. This isn't the first time I'd seen GraphQL, but it was the first time I'd seen GraphiQL, an in-browser tool for building and executing GraphQL queries. Basically it introspects the GraphQL schema from a server and provides auto-complete to build a query, then you can execute the query and see the results instantly.

The speaker also tailored how he continues to work on a Drupal module to expose Drupal data via GraphQL. It sounds like the project is at a minimum viable state and has room for more contributors.

I love the idea of GraphQL and believe it can supplant REST as the best way to expose data, at least for web applications:

• The shape of the response matches the shape of the query, making it more predictable to parse
• It's backed by a spec protocol, meaning any software stack can expose its data as long as it fits the schema format, and any client stack that speaks HTTP can send GraphQL queries

Elm - Frontend with Guarantees

Elm is a functional language for rendering browser-based user interfaces. I have been hearing about it in bits and pieces over the last year and I continue to be enticed by it.

Code speaks louder than any description at this point, so check out this basic counter built with Elm.

Elm is statically-typed and compiles to JavaScript. That might sound terrible, and it probably is. But what's awesome about it is that data only flows one way within an Elm program: from its source of truth (model) downward into the components whose rendering depends on it (view). If data changes the component is re-rendered again. No need to inspect the current state of the UI component and decide how to change it. The data simply flows thru the render pipeline again to get the new result. If this idea sounds just like those in React that's because it is.

One-way data flow and re-rendering components will be core features of the future of building user interface applications on the web. Distributed state is tough and we're only now seeing quality languages and tools for managing it. Elm is a language tailored to that paradigm.

Event: Drupal Trivia

On the last night of the conference, after the closing keynote, and after the event mourning the death of Drupal 6, there was a trivia night held at the World War II Museum in New Orleans. What a fantastic venue! Several planes from the World War II era hung ominously suspended above 100 small round tables.

Each table composed of 5 people made up a team. The trivia panel entertained us with five rounds of trivia questions, ranging from very Drupal specific ("Name the date when Drupal 8.0.0 was released. NO, you do NOT get extra points for giving the exact UTC timestamp or its git SHA1, you nerds!") all the way to nuanced browser questions ("What is the maximum number of stylesheets Internet Explorer 10 allows? Is it A, 1023; B, 4095; C, 8190.") or needing to identify a sponsor company's name by only its logo.

Trivia was a very fun and lively atmosphere, and reminded me of Jeremy Mikola's PHP Jeopardy events at other PHP conferences. More conferences should do this because we're all geeks—we love sharing the small tidbits we pick up about our favorite languages, frameworks and technology. Trivia night gives us a chance to be rewarded for this useless knowledge all while growing our camaraderie of dissenters.

Conclusion

Most conferences I've attended put most of their emphasis on the talks. What's different about DrupalCon is their focus on community and events. Yes the talks are good, but it's all the supplemental activities packed into the schedule that opened my eyes to what's possible during a conference:

• A full day of trainings and summits before the main conference begins
• A well-produced theatrical music production with elaborate costumes and lines and musical numbers and dances
• Three solid keynotes: one technical, one inter-personal/social, and one personal wellbeing.
• Three days and eight tracks (!) of unconference meetings. People can share a similar interest and informally get together to talk and discuss. DrupalCon calls these "Birds of a Feather," or BOF for short.
• Three days talks with the ability to choose between eleven different talks per slot
• Lively after-parties after each day of the conference filled with music and dancing
• One day of "Sprints" where attendees contribute back to the Drupal ecosystem for both technical and non-technical people.

The Drupal community has blown me away. I'm astounded at their inclusiveness and openness across personalities and technical abilities. It's no wonder this conference and its community are one of the most diverse tech communities still going strong.

I've long felt Drupal 8 could take the Drupal ecosystem to the next level. The momentum of the Drupal ecosystem over the years has now met with more widely adopted technical practices of the greater software community. It has matured into a rich platform ready to move into the enterprise professional development market. After this conference all of those feelings were confirmed.

Professionally I'm in a fantastic spot to benefit from developing on top of Drupal 8. I've been using the Symfony components that power Drupal since their inception 5 years ago, and since then have been building web applications using all the principles and tools Drupal has only recently adopted. There is tremendous immediate opportunity in the Drupal ecosystem for experienced PHP developers to find high-demand, well-paid work. It's in my best professional interest to learn more about exactly how people are using Drupal today, and begin consulting with teams wanting to adopt Drupal 8.

Attending and speaking at this DrupalCon solidified for me that the Drupal ecosystem is one of PHP's biggest assets and the Drupal platform is a very viable solution for certain types of software.

It's not only the technology that will continue to keep Drupal relevant for the coming years but its tirelessly passionate community of diverse technical talent. Having embraced a more modern and technically sound infrastructure in the last few years Drupal is now at the perfect cross-section of technical potential and bolstering community to foster huge growth opportunities in the coming years.

Thank you DrupalCon organizers, volunteers, speakers, sponsors and attendees for making my first DrupalCon truly memorable. I look forward to seeing you again soon!

Cosmos: A Spacetime Odyssey

Glow-in-the-dark stars dot the dark ceiling above my top bunk bed, my brother sleeping below me. Space has fascinated me since I was a kid. Science was cool, and I learned early in life that the mysterious world around me could be understood through observation and experimentation.

Between late high school and early college I lost touch with science. The immediacy of computers and technology gripped me and I had to understand them deeper. I don't regret those years for they birthed my programming career. It wasn't until after college in my mid-20's, amid much self-reflection and reevaluation, that I space and science found their way back to me.

It was 2009 when I first discovered Carl Sagan's TV series, Cosmos: A Personal Voyage (1980). (The entire series is on YouTub or Hulu. Start here.) Carl Sagan's way of confidently asserting the seemingly unknowable captivated me. His poetic words stuck with me long after I first heard them. Repeat viewings have instilled a framework for making sense of myself and my insignificant place in the universe.

When news broke in late 2011 that Cosmos was being reborn with new episodes, I was overwhelmed with excitement. Finally, the ideas that inspired me so deeply might inspire a new generation and update with the latest findings in the 30-some years since the original.

Cosmos: A Spacetime Odyssey (2014) is hosted and narrated by astrophysicist Neil deGrasse Tyson. Writers for the original Cosmos, Ann Druyan and Steven Stower, are writing the new series. Music by Alan Silvestri, who scored the movie Contact based on Sagan's novel of the same name, is on board to score the series.

The show and its writing pay tribute to the original by sprinkling oft-quoted clips of Sagan, and reuse metaphors such as The Spaceship of the Imagination, The Cosmic Calendar and the Evolution in 40 Seconds animation. The visuals effects rival that of modern blockbuster films. Animation segments exploring historic scientific figures are stylish yet original and take what could be yet another boring historical vantage point and make it very engaging.

Realizing the Cosmos's ultimate goal would mean viewers rethinking what they believe to be true about who they are, where they come from and where they're going.

Cosmos currently premieres new episodes on Sundays at 9:00pm Eastern on Fox. Previous episodes can be viewed online at CosmosOnTV.com and on Netflix.

This post may now be outdated. See the johnkary/phpunit-speedtrap project on GitHub for the most recent instructions.

All developers should strive to write "fast tests," and "fast" will be different for each developer, language and software stack. Traditionally, unit tests should be very fast, a few milliseconds or faster, while functional tests and end-to-end tests will run a bit longer, maybe a few seconds.

As your test suite grows, you can expect it to run longer. But if you're not careful, some slower tests can creep in and disproportionately increase your suite's total execution time.

SpeedTrap

SpeedTrap is a PHPUnit Listener that reports on slow-running tests in your PHPUnit test suite console output.

Many factors affect test execution time. A test not properly isolated from variable latency (database, network, etc.) and even basic load on your test machine will cause test times to fluctuate.

SpeedTrap helps you identify slow tests but cannot tell you why those tests are slow. For that you should check out something like PHPUnit_Listener_XHProf to help identify specifically which methods in your call stack are slow.

Installation

SpeedTrap is installable via Composer and should be added as a dev dependency:

composer require --dev johnkary/phpunit-speedtrap

Usage

Enabled it with default configuration by adding the following to your test suite's phpunit.xml file:

<phpunit bootstrap="vendor/autoload.php">
...
    <listeners>
        <listener class="JohnKary\PHPUnit\Listener\SpeedTrapListener" />
    </listeners>
</phpunit>

Now run your test suite as normal. If tests run that exceed the slowness threshold (500ms by default), SpeedTrap will report on them in the console after the suite completes.

Configuration

SpeedTrap has two configurable parameters:

• slowThreshold - Number of milliseconds a test takes to execute before being considered "slow" (Default: 500ms)
• reportLength - Number of slow tests included in the report (Default: 10 tests)

These configuration parameters are set in phpunit.xml when adding the listener:

<phpunit bootstrap="vendor/autoload.php">
    <!-- ... other suite configuration here ... --></p>

<pre><code>&lt;listeners&gt;
    &lt;listener class="JohnKary\PHPUnit\Listener\SpeedTrapListener"&gt;
        &lt;arguments&gt;
            &lt;array&gt;
                &lt;element key="slowThreshold"&gt;
                    &lt;integer&gt;500&lt;/integer&gt;
                &lt;/element&gt;
                &lt;element key="reportLength"&gt;
                    &lt;integer&gt;10&lt;/integer&gt;
                &lt;/element&gt;
            &lt;/array&gt;
        &lt;/arguments&gt;
    &lt;/listener&gt;
&lt;/listeners&gt;
</code></pre>

<p></phpunit>

This allows you to set your own criteria for "slow" tests, and how many you care to know about.

Inspiration

SpeedTest was inspired by Rspec's -p option that displays feedback about slow tests.

License

phpunit-speedtrap is available under the MIT License.

When registering my account on this specific site, I randomly generated a long, complex password and copied it to my clipboard with the intent to paste it into the password and password-confirmation text inputs.

I placed my cursor in the password box and pressed "⌘ + V" to paste my clipboard text into the input. Nothing. Maybe I didn't copy the password correctly. I generated another password and pasted again. Nothing. Ah great, another site preventing its users from pasting text into text boxes.

I tweeted the following, partly wondering allowed, partly in frustration:

Why do some sites disallow pasting from clipboard into text inputs? I really don't want to type my 32-character randomly generated passwords

— John Kary (@johnkary) January 5, 2014

Site features should be oriented towards business goals, right? So what business goal might a company have with not allowing a user to paste text?

I image the business saying their goal was "to decrease incorrect registration data, resulting in less user frustration and less required user support." Sounds good in theory... Keep users happy when using the site, and keep company expenses down by requiring less staff to support users misusing their software.

Balancing the sliding scale

Implementing information security is a trade-off between convenience and security of data. This almost always functions as a sliding scale: practices that are easier to use are often less secure. But the most secure actions often take much more effort on behalf of the user. Too much inconvenience and the user will forgo implementing security all together.

When I think of "online security" I think of things like encryption, anonymity and privacy. The last piece, "privacy," is usually implemented as segregation of data access, implemented via user account and password.

If an account can only be accessed by one who knows its password, data is reasonably more secure than if it were publicly available. (The inconveniences in using other stronger authentication mechanisms have probably prevented widespread adoption for most consumer-facing applications.) So in order to use online accounts with some added privacy, users tolerate the inconvenience of creating a new user account and password for each site they want to access privately.

Solving the wrong problem

Most sites require entering a username and entering the same password twice when creating an account.

The current standard practice to decrease erroneous password entry is requiring the user to enter their password twice: once in two separate text inputs. A password like il0vemydog123! isn't too inconvenient to type two times on a full sized keyboard, and is perhaps mildly inconvenient to type twice on a mobile phone keyboard. The trade-off is the inconvenience of typing a password twice is less inconvenient than if the user were to mistype their password once and become locked out of their account.

Yet I firmly believe password confirmation boxes are solving the wrong problem. The only reason a user must type a password twice is to decrease the likelihood they mistyped their intended password one of the two times. Why might they mistype it? Because the user can't see the result of their typing.

I side with Luke Wroblewski on how to solve this issue:

1. Password boxes should offer a "Hide/Show" button adjacent to a single password box that obscures the user's typing by replacing each character with an asterisk.
2. Using this strategy, the user's password is typed in plain-text and is visible by default, leading to visual confirmation that the password was typed correctly the first time.

Mobile devices continue to increase in penetration, where typing is much more difficult, so developers should firstly reevaluate this long-held and broken user experience practice of requiring a user to confirm their password using two password boxes.

Luke does a great job illustrating the pros and cons of this approach in his post Mobile Design Details: Hide/Show Passwords.

Disabling pasting text could decrease password security

My friend and go-to security expert, Bryan Geraghty, offered this insight in response to my earlier tweet:

@johnkary Not allowing users copy and paste passwords means that you are likely an easier brute-force target. We don't report that ATM.

— Bryan C. Geraghty (@archwisp) January 5, 2014

Why might this be the case?

A long, complex password is inherently difficult to type. If a user must type a complex password twice, there are probably a certain percentage of users that will forgo typing a long password and either use a shorter password or one that requires less key-presses to type.

This is where Bryan's point aligns with the security sliding scale: a user experience that makes it more difficult to use a better security practice will likely lead to a choice that is easier for the user and less secure for their data. In this case, a weaker password is easier to guess or extract via a brute force attack.

Working around the inability to paste

Sites that disable the ability to paste do this with JavaScript, often by listening for the paste browser Event, then preventing the default behavior that would normally take the clipboard text and place it at the cursor location. Less sophisticated preventions might simply intercept a platform-specific key combination.

There are several ways we can work around this restriction by executing our own JavaScript:

1. Use a password vault browser plugin with "auto-fill" capability
2. The browser's Location bar prefixed with javascript:
3. The browser's Developer Console
4. A custom bookmarklet
5. Write a custom browser extension

Password Vault Browser Plugin

Password vaults like LastPass or 1Password have browser extensions that will fill in a saved username and password into a site's text inputs for user login. These extensions use JavaScript and a stored input selector associated with a site/URL to target each field that must be filled.

These extensions are your best bet for security by…

1. Preventing you from ever needing to manually type your password (preventing keylogger interception)
2. URL/domain matching prevents entering credentials into phishing sites appearing as legitimate sites.

If you're not using a password vault yet, I highly encourage you to start today! If you're not yet ready for that commitment, keep reading. We can still circumvent sites preventing you from fully using their registration form.

Custom JavaScript solutions

The remaining four solutions require us to execute custom JavaScript we write ourselves. This vanilla JavaScript will force our own text value into a text input.

But first you need to find a way for JavaScript to select the target text input. There are several ways to select the element (id, class, name), but the easiest is by its HTML "id" attribute via getElementById(). If your site requires a different solution, I will leave that to you to investigate.

document.getElementById('password').value = 'mypassword';

We can execute this code on the page in a few ways.

Location Bar

The Location bar can execute JavaScript when code is prefixed with javascript::

javascript:function fillpw()%7Bdocument.getElementById('password').value='mypassword';%7Dfillpw();

The downsides of this approach are that this text could be sent to the browser vendor (via auto-complete engine or other statistics-gathering techniques), submitted to the browser's default search engine if mistyped, or persisted in the Location bar's plain-text history.

At the risk of exposing your password in persisted plain-text, there are better solutions.

Developer Console

If you only need to override the ability to paste just one time, the following technique is probably your next best option after the password vault technique above.

Open the developer console using its keyboard shortcut (Chrome, Firefox) or menu selection, and enter the following JavaScript:

document.getElementById('password').value = 'mypassword';

Custom Bookmarklet

If you have one offending site that you repeatedly visit, you'll want a more dynamic solution. A bookmarklet stored in your browser's Bookmarks Bar can execute JavaScript, just like the "Location Bar" suggestion above.

The rough steps go something like so:

1. Create a bookmark by dragging a link to your browser's Bookmarks Bar.
2. Edit the bookmark you just created and enter the following JavaScript, replacing the element's id with your offending site's input id:

javascript:function fillpw()%7Bdocument.getElementById(prompt('Password element id:')).value=prompt('Password:');%7Dfillpw();

Using this solution, you can click the bookmarklet and be prompted for the HTML id of the text input you would like to modify its value, then prompted for the text you would like to enter. Entering via a JavaScript prompt prevents your password from being stored in plain-text at any point in the process.

You can now navigate to the offending site, click the bookmarklet in your bookmark bar and enter in your credentials.

Custom Browser Extension

If there are many offending sites you must visit, you could expand the "Custom Bookmarklet" solution with more code that switches on the current document.location.host or document.location.href to target the correct username and password widgets for a given URL.

If you're considering this solution, you're probably much better off by adopting a password vault that already has this functionality built-in. That is unless you really want to learn how to build a browser extension.

Each browser has its own conventions for writing extensions or add-ons, and exactly how to do that will be outside the scope of this post, but check the following resources:

• Getting Started Building Chrome Extensions
• Building a Firefox extension

Wrap up

It should be your responsibility as an informed developer that the next time you're tasked with creating a registration or login form, to instead opt for a single, visible-password entry widget that offers a corresponding button to obscure the user's typing.

When creating your next registration form I suggest following the implementation details in Luke Wroblewski's post: Mobile Design Details: Hide/Show Password.

Let's continue making the web more friendly and easier to use, and lead our users to embrace better security practices.

This post is in no way endorsed or sponsored by the companies whose products are mentioned. These are my own uninfluenced thoughts.

I discovered Tom Bihn bags from a post on the Rands in Repose blog. A lot like me, its author is a bag geek. I devoured all the different scenarios and reasons he is picky in selecting a bag. I nodded my head at all of his points, and by the end was picking apart my existing backpack and immediately had a bad case of bag envy. I was sold on the thought of a new bag, and behold, his post linked off to a few possibilities.

Before picking up the Tom Bihn Buzz Sling Bag, I must admit I was guilty of carrying a bulky backpack. I didn't have much of a choice when my previous laptop was a behemoth 17" MacBook Pro, but when I downsized to a 13" MacBook Air in Summer 2012 I wanted a bag that also matched the streamlined nature of the Air.

Bag Requirements

My (new) criteria for a daily computer bag includes the following:

• Low profile
• Streamlined look
• Big enough to store a MacBook Air and charger with some room to spare
• Easy on-off
• No useless straps
• Sturdy, can take some punishment
• Not scream NERDY BACKPACK

The Tom Bihn site has a limited selection of bags, making it easy to pick one simply based on core bag size. Choosing the Buzz bag over the others was an easy choice.

Impressions

I could weave these thoughts into prose but they are fairly easy to skim:

• Shoulder Strap - well formed and fit for the bag. It's wide and sturdy enough, but relatively thin in its thickness, so it doesn't add much bulk when slung over the shoulder and chest.

• Shoulder Strap - a little uncomfortable at first and stiff out of the box. I was hoping for thicker padding on the side against the body. When fully loaded (laptop + charger + a thick book + all random cables and gadgets) it weighs unevenly on my shoulder and causes some discomfort.

• Shoulder Strap Buckle - a heavy duty plastic buckle fastens the upper strap to the lower strap near the base of where the strap affixes to the bottom of the bag. The buckle is solidly built and makes a satisfying click sound when fastening. The buckle makes it extremely easy to slip out of the bag without lifting it over your head.

• Shoulder Strap Thumb Loop - There's a sturdy loop near the phone/wallet pocket on the shoulder strap to insert your thumb and support the weight of the bag after unlatching the buckle. The placement and size of the loop feels very natural when using it. It is easy to then sling the bag somewhere or hold it up to reattach the plastic buckle.

• Shoulder Strap Contour - At first the bag didn't feel secured to my body no matter what length I extended or retracted the shoulder strap. After a few months of repeated daily use the strap gave a little and felt more contoured.

• Shoulder Strap Pocket - Perfectly cradles my iPhone 4S (with thin case) or my Slimmy wallet. It has enough padding to where I wouldn't be afraid putting it in there and flinging my bag around under normal circumstances. But I find it's my permanent home for my wallet when on the go with my bag.

• Shoulder Strap Pocket - Pocket flip-cover is just the right size to still allow a set of normal earbuds to plug into the top of an iPhone 4S or 5 while secured in the pocket.

• Side Pocket - A tight fit to accommodate my admittedly large 750mL Nalgene water bottle. The pocket looks like it's more made for a thin aluminum water bottle or small travel umbrella.

• Zippers - Two words: HEAVY DUTY. They almost seem to be more heavy duty than a low-profile bag should be. They are perhaps the most durable piece of the bag and I could see them continuing to zip perfectly even after years of use. Not to mention they have a waterproof lining over them. If I were caught in a moderate rain shower I wouldn't be fearful of water seeping in.

• Zippers - Both outside pockets offer dual-zippers that zip both ways.

• Waist Strap - Would be useful if I planned on using the bag during a more active activity like hiking. The straps hang too low off the back when not in use to tolerate it being attached at all times. Not to mention it looks sloppy hanging off the bag, and goes against my "no useless straps" requirement. My primary use case is transporting my laptop between home and destination so I removed it. It unbuckles after some moderate pressure on each of the metal-hinge plastic connections to the bag.

• Large Pocket - Has a great padded separator between the laptop compartment and the main bag compartment. My 13" MacBook Air with Encase padded sleeve comfortably fits in the bag. It could easily accommodate a 15" MacBook Pro, too. The other side of the padded separator is a great place to stash larger cables, a tablet, a medium sized book (400 pages, hardback) or even a second laptop. Even with those compartments full, the bag does not appear bulky.

• Small Pocket - Well-angled slots to add pens. They slip in almost sideways and fit like a glove with two normal-sized pens in each slot. There is a bigger pouch for other things, but it feels awkwardly sized. Perhaps it's meant to fit a normal-sized wallet, but I find it's not well secured. I can place a pack of gum or chap stick in there but might find them removed from where I packed them when I arrive. There is also a 6-inch strap with clips on both ends: one to affix it to the bag, one to easily clip your keyring so it doesn't get lost in the clutter.

• Flying - When flying with the bag it works perfect as a "personal item" by fitting up under the seat with plenty of room to relax your feet on the bag.

What would I change?

I wish the very front of the bag had a stash pouch for small items such headphones, pack of gum, chap stick, nail clippers, etc. Instead I keep several Ziplock bags, one with cables and connectors, another with over the counter medications and similar. I can open the full front zipper pocket and dig deep into the bag without things tumbling out, but wish it was easier to store things more quickly.

Final Verdict

Overall I'm still very happy with this bag. Its streamlined look and low profile make it great for a daily carrier from point A to B or even when flying. It doesn't offer the quick-access pouch I came to rely on with previous bags, but this bag is a keeper and will be my primary bag for years to come.

Sadly this bag design has been "retired" and is no longer available for purchase from the vendor. But if this bag is any indication of the quality of their other bags, I wouldn't hesitate to buy another model.

This post was published when Symfony Standard Edition 2.3 was the latest stable release, and may be outdated for later Symfony versions.

Have you explored all of the Console commands that ship with the Symfony Standard Edition? The long list of tasks after entering the php app/console command might feel a bit overwhelming at first, but the more you use Symfony, the more comfortable you'll feel and the more likely you will have memorized a few common commands.

But there are a few commands that you don't hear much about that are very useful during every day development, but only garnish a quick blurb in the official documentation.

I spend most of my working day developing web applications where the frontend JavaScript application is fed data by an API built on the Symfony2 framework. What follows are some of the protips I've devised to help me work more efficiently and some of the non-obvious Symfony commands I use on a daily basis. I hope they help you develop faster and more efficiently!

1. Alias your most common Symfony commands
   1. Alias the Symfony Console itself
   2. Alias the cache:clear command
   3. Alias the command to run your test suite
   4. Find other common commands you can alias
2. Finding a specific service id or a service's class
3. Finding a specific container parameter
4. Finding a specific route name or URI
5. Show all info about a specific route by its name
6. Find which route matches a given URI
7. Query your database from the CLI
   1. DQL queries
   2. SQL queries

1. Alias your most common Symfony commands

There are 3 commands I use so often I have short bash aliases setup for each:

Alias the Symfony Console itself

Developing with Symfony2, you will type php app/console many, many times. Having a faster way to type this is an instant productivity boost.

Add the following line to your .bash_profile file in your home directory to take your first step towards Symfony console mastery:

alias sf='php app/console'

Now reload your .bash_profile within you current shell session with the command source ~/.bash_profile and you can now access the Symfony console by typing sf in all places you would type php app/console.

Try using your new powers and ensure they're working properly:

$ sf
Symfony version 2.3.7 - app/dev/debug</p>

<p>Usage:
  [options] command [arguments]</p>

<p>Options:
  --help           -h Display this help message.
  --quiet          -q Do not output any message.
...

Alias the cache:clear command

You will clear the Symfony cache relatively often, so it deserves its own shell alias. As many will tell you, step 1 of troubleshooting Symfony issues is first clearing your cache!

alias sfcc="php app/console cache:clear"

The command sfcc will now clear your "dev" environment cache. The --env=dev option is implied for all Symfony commands unless otherwise specified, but you can easily specify any other environment too:

$ sfcc --env=test

Alias the command to run your test suite

You should be running your test suite often during development. The full command to run the Symfony test suite using PHPUnit is a bit long:

$ bin/phpunit -c app

Not too bad, but again, given how often you will be running this command, it deserves its own alias. (If you don't use PHPUnit, modify the alias to use whatever your test runner requires):

alias sft='bin/phpunit -c app'

Quickly reload your .bash_profile again via source ~/.bash_profile and you can now execute your test suite by typing sft:

$ sft
PHPUnit 3.7.28 by Sebastian Bergmann.</p>

<p>Configuration read from /path/to/your/project/app/phpunit.xml.dist</p>

<p>...............................................................  63 / 120 ( 52%)
.........................................................</p>

<p>Time: 312 ms, Memory: 5.75Mb</p>

<p>OK (120 tests, 236 assertions)

With your new-found powers you can now quickly do things like only run tests within a specific directory:

$ sft src/Acme/DemoBundle/

Or run a specific test by its name:

$ sft --filter=testUserStartsAsActive

Or quickly generate code coverage:

$ sft --coverage-html=coverage

Of course you could always run these exact commands without the alias, but the key is allocating less mental energy to mundane things like typing long commands for commonly used commands.

Find other common commands you can alias

You could try noticing your habits and alias other common commands, but why not let your console history just tell you?

$ history | grep 'php app/console'
[... your most recent history containing 'php app/console' will appear here ...]

2. Finding a specific service id or a service's class

Often you will need to see if your newly defined service is wired up how you intend it to be. The following command will display all defined services in the Symfony2 service container:

$ sf container:debug

If you have an idea of what you're looking for, pipe this command's output to grep <you search string here>. For example, let's list all the services containing the word "doctrine":

$ sf container:debug | grep doctrine
database_connection                                    n/a       alias for doctrine.dbal.default_connection
doctrine                                               container Doctrine\Bundle\DoctrineBundle\Registry
doctrine.dbal.connection_factory                       container Doctrine\Bundle\DoctrineBundle\ConnectionFactory
doctrine.dbal.default_connection                       container stdClass
doctrine.orm.default_entity_manager                    container EntityManager5292d7ee8f526_546a8d27f194334ee012bfe64f629947b07e4919__CG__\Doctrine\ORM\EntityManager
doctrine.orm.default_manager_configurator              container Doctrine\Bundle\DoctrineBundle\ManagerConfigurator
doctrine.orm.entity_manager                            n/a       alias for doctrine.orm.default_entity_manager
doctrine.orm.validator.unique                          container Symfony\Bridge\Doctrine\Validator\Constraints\UniqueEntityValidator
doctrine.orm.validator_initializer                     container Symfony\Bridge\Doctrine\Validator\DoctrineInitializer
form.type_guesser.doctrine                             container Symfony\Bridge\Doctrine\Form\DoctrineOrmTypeGuesser
monolog.logger.doctrine                                container Symfony\Bridge\Monolog\Logger
sensio_framework_extra.converter.doctrine.orm          container Sensio\Bundle\FrameworkExtraBundle\Request\ParamConverter\DoctrineParamConverter

This command also accepts a service id as its first argument and will output even more information about that how it is wired up. Let's look at the service with id "twig.loader":

$ sf container:debug twig.loader
[container] Information for service twig.loader</p>

<p>Service Id       twig.loader
Class            Symfony\Bundle\TwigBundle\Loader\FilesystemLoader
Tags
    - twig.loader                    ()
Scope            container
Public           yes
Synthetic        no
Required File    -

(I wish this output contained things like what other services/parameters are wired into it, what factory methods are run when instantiated, etc. If you know how to find this output from the Console, please comment below.)

3. Finding a specific container parameter

While sf container:debug outputs a list of services, calling this command with the additional option --parameters will dump a list of all service container parameters and their values. This is useful to ensure your config or bundle is overriding the intended parameter or just to see what all parameters are available without digging into a Bundle's Extension or Configuration files:

$ sf container:debug --parameters

Using this command's output with grep, like in the previous example, is also your friend.

You can also use the similar option --parameter="..." (notice no "s" in "parameter") to find the value for a specific parameter. Let's find exactly which class will be used for Doctrine's EntityManager by finding the value for the "doctrine.orm.entity_manager.class" parameter:

$ sf container:debug --parameter="doctrine.orm.entity_manager.class"
Doctrine\ORM\EntityManager

4. Finding a specific route name or URI

Like finding values in the Service Container, a similar command exists to search Symfony's Router for route names and URI paths:

$ sf router:debug
[router] Current routes
Name                Method Scheme Host Path
index               ANY    ANY    ANY  /
users_list          ANY    ANY    ANY  /users/
users_new           ANY    ANY    ANY  /users/new
users_edit          ANY    ANY    ANY  /users/{id}

I often use this command as a dummy-check that my Controller @Route annotations are wired properly and that they have the correct @Method annotation values.

5. Show all info about a specific route by its name

The first optional argument to router:debug is the route name. This shows all details for the route named "users_list":

$ sf router:debug users_list
[router] Route "users_list"
Name         users_list
Path         /users/
Host         ANY
Scheme       ANY
Method       ANY
Class        Symfony\Component\Routing\Route
Defaults     _controller: Acme\DemoBundle\Controller\UsersController::indexAction
Requirements NO CUSTOM
Options      compiler_class: Symfony\Component\Routing\RouteCompiler
Path-Regex   #^/users/$#s

6. Find which route matches a given URI

You might know the URI path but not the route name. The router:match command works like router:debug but does the reverse operation by accepting a URI argument instead of the route name.

The following example shows which route matches the URI "/users":

$ sf router:match /users
Route "users_list" matches</p>

<p>[router] Route "users_list"
Name         users_list
Path         /users/
Host         ANY
Scheme       ANY
Method       ANY
Class        Symfony\Component\Routing\Route
Defaults     _controller: Acme\DemoBundle\Controller\UsersController::indexAction
Requirements NO CUSTOM
Options      compiler_class: Symfony\Component\Routing\RouteCompiler
Path-Regex   #^/users/$#s

7. Query your database from the CLI

It can be a hassle to write up a functional test or other code that executes DQL or SQL you are tweaking for a new query or troubleshooting an existing query. Thankfully Doctrine offers two commands that let you run queries straight from the command-line.

DQL queries

You can query your database using raw DQL right from the command-line using the command doctrine:query:dql. Just pass in the full DQL you want to execute:

$ sf doctrine:query:dql "SELECT u.firstName, u.lastName, u.lastLogin FROM AcmeDemoBundle:User u" --hydrate=array</p>

<p>array (size=1)
  0 =>
    array (size=3)
      'firstName' => string 'John' (length=4)
      'lastName' => string 'Smith' (length=5)
      'lastLogin' =>
        object(stdClass)[325]
          public '<strong>CLASS</strong>' => string 'DateTime' (length=8)
          public 'date' => string '2013-11-26T00:00:00-06:00' (length=25)
          public 'timezone' => string 'America/Chicago' (length=15)

Appending the option --hydrate=array like in the previous example is also useful if your selected entities have other entity relationships, where the resulting var_dump() of object data creates a lot of additional clutter. Hydrating as an array is a bit cleaner to look at if you're only interested in the data and not the underlying object structure.

SQL queries

You can also directly query your database using raw SQL without connecting through the mysql binary or a GUI application. Like querying for DQL, the connection data from parameters.yml or config.yml is used, so there's no hassle with passing the right credentials with each query:

$ php app/console doctrine:query:sql "SELECT * from users"</p>

<p>array (size=1)
  0 =>
    array (size=5)
      'id' => string '5' (length=1)
      'firstName' => string 'John' (length=4)
      'lastName' => string 'Smith' (length=5)
      'logins' => null
      'lastLogin' => string '2013-11-26' (length=10)

Did I miss anything?

What do you think? Did you learn something that can make you more productive? Are there other productivity wins you wish I had included? I'd love to hear your feedback in the comments section below!

Note: This post was published when Symfony Standard Edition 2.3 and jms/serializer 0.12.0 were the latest stable releases, and may be outdated with later versions.

The JMS Serializer library and its accompanying Symfony2 bundle are must-haves if your PHP-based application needs to expose an HTTP API. If you have used it before, you're probably most familiar with its ability to transform an object graph to XML or JSON, and this couldn't be easier in Symfony2:

$this->get('jms.serializer')->serialize($user, 'json');

The resulting JSON might look something like this:

{
   "first_name":"John",
   "last_name":"Kary",
   "logins":5,
   "last_login":"2013-07-21T01:16:22-0500",
   "address":{
      "number":"555",
      "street":"4th Street",
      "city":"Lawrence",
      "state":"KS",
      "zip":"66045"
   }
}

What I've seen less of, and what this post will highlight, is the Serializer library's ability to take XML (and JSON) and transform it back into an object graph.

Data exchange formats and composite data types

Serializing PHP objects to JSON or XML is painless because the rich data types encapsulated by a PHP object are transformed from their PHP data constructs, which allow composing rich "composite types" of data (e.g. your domain objects) made up of primitive types (e.g. strings, integers, booleans, etc), into a format like JSON or XML that is limited to only primitive data types.

Having a small number of primitive data types is an advantage of a data exchange format like JSON or XML. They are easy to create and allow high-level languages to work with data in their native language constructs before decomposing that data back into a data exchange format to allow another application to use it.

To take advantage of the exchange format and rebuild our feature-rich PHP objects, we must provide the Serializer service more explicit details about what data types our domain objects expect each property to use.

You've probably seen PHPDoc annotations like @var string or @var \DateTime on PHP objects. phpDocumentor and most IDEs will parse and use these for data completion, but the Serializer component uses its own annotations that provide a bit more detail.

Annotating object properties with Serializer Types

Your Acme\DemoBundle\Entity\User object probably has a lot of things it can do in the form of methods, and likely has several relationships to other objects. We want to take advantage of that rich domain model and the behavior of those objects once we receive data from our third-party API. Not only that but we want to hydrate those model objects without writing a bunch of boiler-plate code.

With a little additional configuration, we can allow the JMS Serializer to take JSON or XML and properly deserialize it back into PHP domain objects.

Here's our basic User object. I have omitted all of the normal behavior methods and all getters/setters for brevity, so we'll just use public properties. Chances are your User object would have a lot more methods and use private properties exposed by getters and setters.

namespace Acme\DemoBundle\Entity;</p>

<p>use Doctrine\Common\Collections\ArrayCollection;</p>

<p>class User
{
    public $firstName;
    public $lastName;
    public $logins;
    public $lastLogin;
    public $address;
    public $friends;</p>

<pre><code>public function __construct()
{
    $this-&gt;address = new Address();
    $this-&gt;friends = new ArrayCollection();
}
</code></pre>

<p>}

Let's say your company stores its user data on another server and exposes it via an HTTP API to your custom application. You decide to fetch an XML representation of that user resource via a request GET http://someapi.com/users/10 and the response body might look like this:

<?xml version="1.0" encoding="UTF-8"?>
<result>
  <first_name>John</first_name>
  <last_name>Kary</last_name>
  <logins>5</logins>
  <last_login>2013-07-21T00:55:59-0500</last_login>
  <address>
    <number>555</number>
    <street>4th Street</street>
    <city>Lawrence</city>
    <state>KS</state>
    <zip>66045</zip>
  </address>
  <friends>
    <friend>
      <first_name>Chris</first_name>
      <last_name>Escalante</last_name>
      <logins>3</logins>
      <last_login>2013-07-11T12:22:41-0500</last_login>
    </friend>
    <friend>
      <first_name>Dustin</first_name>
      <last_name>Gordon</last_name>
      <logins>1</logins>
      <last_login>2013-06-26T14:16:33-0500</last_login>
    </friend>
  </friends>
</result>

Given this response, you probably thought you would try using JMS Serializer to deserialize this XML into a User object. You probably discovered the SerializerInterface->deserialize($data, $type, $format) method, passed in your XML data, provided the fully-qualified class name to your target object and told it to use the XML format:

$this->get('jms.serializer')->deserialize($xml, 'Acme\DemoBundle\Entity\User', 'xml');

If you tried this as-is you would be met with a JMS\Serializer\Exception\RuntimeException with an exception message like this:

You must define a type for Acme\DemoBundle\Entity\User::$firstName.

We must tell the Serializer what format or data type to use when translating an XML node like <first_name>John</first_name> back onto our User object's property public $firstName.

Let's annotate our properties to parse this XML. Be sure to reference the official Annotation documentation for all possible annotations:

namespace Acme\DemoBundle\Entity;</p>

<p>use Doctrine\Common\Collections\ArrayCollection;
use JMS\Serializer\Annotation as JMS;</p>

<p>class User
{
    /<em>*
     * @JMS\Type("string")
     </em>/
    public $firstName;</p>

<pre><code>/**
 * @JMS\Type("string")
 */
public $lastName;

/**
 * Number of times has logged in.
 * @JMS\Type("integer")
 */
public $logins;

/**
 * Last time logged in using ISO 8601 format.
 * @JMS\Type("DateTime&lt;'Y-m-d\TH:i:sO'&gt;")
 */
public $lastLogin;

/**
 * @JMS\Type("Acme\DemoBundle\Entity\Address")
 */
public $address;

/**
 * @JMS\Type("ArrayCollection&lt;Acme\DemoBundle\Entity\User&gt;")
 * @JMS\XmlList(entry="friend")
 */
public $friends;

public function __construct()
{
    $this-&gt;address = new Address();
    $this-&gt;friends = new ArrayCollection();
}
</code></pre>

<p>}

The annotations we're using above:

• @Type
• @XmlList

We can now take that XML response and deserialize it back into a rich User object. Here's the full code embedded in a controller where you might make your request to the HTTP API and do something with the XML response:

namespace Acme\DemoBundle\Controller;</p>

<p>use Symfony\Bundle\FrameworkBundle\Controller\Controller;
use Sensio\Bundle\FrameworkExtraBundle\Configuration\Route;</p>

<p>class DemoController extends Controller
{
    /<em>*
     * @Route("/", name="deserialize")
     </em>/
    public function helloAction()
    {
        // Assume you get your XML from some third-party
        $xml = &lt;&lt;<EOT
<?xml version="1.0" encoding="UTF-8"?>
<result>
  <first_name>John</first_name>
  <last_name>Kary</last_name>
  <logins>5</logins>
  <last_login>2013-07-21T00:55:59-0500</last_login>
  <address>
    <number>555</number>
    <street>4th Street</street>
    <city>Lawrence</city>
    <state>KS</state>
    <zip>66045</zip>
  </address>
  <friends>
    <friend>
      <first_name>Chris</first_name>
      <last_name>Escalante</last_name>
      <logins>3</logins>
      <last_login>2013-07-11T12:22:41-0500</last_login>
    </friend>
    <friend>
      <first_name>Dustin</first_name>
      <last_name>Gordon</last_name>
      <logins>1</logins>
      <last_login>2013-06-26T14:16:33-0500</last_login>
    </friend>
  </friends>
</result>
EOT;</p>

<pre><code>    $user = $this-&gt;get('jms_serializer')-&gt;deserialize($xml, 'Acme\DemoBundle\Entity\User', 'xml');

    var_dump($user);
    exit;
}
</code></pre>

<p>}

I'm using var_dump here to show exactly how the data appears when hydrated onto the User object:

object(Acme\DemoBundle\Entity\User)[416]
  public 'firstName' => string 'John' (length=4)
  public 'lastName' => string 'Kary' (length=4)
  public 'logins' => int 5
  public 'lastLogin' => 
    object(DateTime)[436]
      public 'date' => string '2013-07-21 00:55:59' (length=19)
      public 'timezone_type' => int 1
      public 'timezone' => string '-05:00' (length=6)
  public 'address' => 
    object(Acme\DemoBundle\Entity\Address)[434]
      public 'number' => string '555' (length=3)
      public 'street' => string '4th Street' (length=10)
      public 'city' => string 'Lawrence' (length=8)
      public 'state' => string 'KS' (length=2)
      public 'zip' => string '66045' (length=5)
  public 'friends' => 
    object(Doctrine\Common\Collections\ArrayCollection)[450]
      private '_elements' => 
        array (size=2)
          0 => 
            object(Acme\DemoBundle\Entity\User)[455]
              ...
          1 => 
            object(Acme\DemoBundle\Entity\User)[454]
              ...

We can now work with our User object and it can boast all of its rich behavior and relationships with other objects!

Well looks like those sites sent so much traffic our 512MB server running Apache + MySQL + WP-Cache still couldn't keep up.

That's impressive considering when the same post was on the front page of /r/programming it did about 20,000 views all together, 15,000 of which during one day. I guess front page of HN at noon CST on a Monday lands you a lot of traffic really fast.

Peace out WordPress

So in the interest of not investing more time battling outdated WordPress versions and a bunch of dynamic functionality I simply didn't need, and a desire to learn a bit more Ruby, I've decided to move this blog and the rest of my personal site to be generated by Jekyll.

This now means all of my posts are written in a text editor, formatted in Markdown and stored in a git repository. Deployment is done via a post-merge hook from my local dev repository to my remote server via rsync.

My content is no longer held hostage, holed up in some remote database. It's in plain-text, in a version control repository, where it belongs.

Migrating WordPress to Jekyll

WordPress has built-in export functionality, which dumps all of its pages, posts and comments to XML. It's conveniently accessible from the admin panel at /wp-admin/export.php.

The Jekyll docs have a section on Blog Migrations. I followed the instructions to exchange my newly-generated wordpress.xml into freshly-minted static HTML pages in a format Jekyll expects.

gem install hpricot
ruby -rubygems -e 'require "jekyll/migrators/wordpressdotcom"; Jekyll::WordpressDotCom.process("wordpress.xml")'

I then leveraged Pandoc to convert the resulting HTML files to straight Markdown.

cd ~/Sites/johnkary.github.com/_posts
pandoc --from=html --to=markdown --no-wrap --output=2009-12-08-my-post-title.md 2009-12-08-my-post-title.html

The Pandoc-generated Markdown files were very accurate for the most part. The only fail was I had to re-split the paragraph text because the original Jekyll import dumped my paragraphs without <p> tags, so Pandoc just merged that text together. This was a result of how the text was stored in the WordPress database. I also had to clean up some poor formatting from some older posts, but nothing major for my 20 blog entries.

Development Mode

When I'm developing on my site, I run the built-in Jekyll web server then load the site in my browser at http://0.0.0.0:4000

cd ~/Sites/johnkary.github.com
jekyll --server --auto

--server tells Jekyll to run its WEBrick web server (instead of just generating the site).

--auto tells Jekyll to watch my source files for changes, and regenerate my site files on save. So if I'm writing a new blog post or editing CSS and save my file, Jekyll regenerates the corresponding static file.

Liquid Templating Language

Jekyll uses the Liquid Templating Language. I felt very at home with it right away because it's basically exactly like Twig Templates. And Twig is inspired by Jinja2 templates, which uses the same syntax. I don't know who was "first" in all of this, but there was no additional learning curve for me in learning the templating syntax.

Syntax Highlighting

Jekyll has support for the great syntax highlighting Python library Pygments, which probably supports every language you know.

It supports PHP, but I had to dig around a bit to find its PhpLexer option startinline. If the startinline option is not given, Pygments will only start highlighting after it finds <?php. I don't use <?php in my code examples, so my highlighting wasn't working until I found this Jekyll issue introducing startinline functionality.

{% highlight php %}
// No source highlighting
$name = $person->getName();
{% endhighlight %}</p>

<p>{% highlight php startinline %}
// Source highlighting without opening PHP tag
$name = $person->getName();
{% endhighlight %}

Redirects From Old Site

A number of my blog posts have been picked up on various link portals over the years, and people regularly find my posts using Google. If I took my new blogging engine and gave no care to redirect to the new post URL I would lose my search engine rankings. Google would likely see my new posts as simply new content with no credibility.

Instead I'm choosing a two-headed approach:

1. Use Apache's .htaccess file to explicitly use 301 Redirects to the new URLs. This will tell Google and other crawlers the page has permanently moved. I have then instructed the Google crawler to re-crawl my site to pickup the new URLs. Given a few weeks to a month, my new URLs should reflect in search engines.
2. Use a Jekyll plugin called alias_generator to generate an index.html page at the old URL which has a meta-redirect to the new page.

Yes, a meta-redirect is not ideal and isn't good practice. In the long-term I may move my site to be powered by GitHub Pages, which doesn't support .htaccess or other means of redirects. It's at this point that anyone still linking to my old posts can still find their way to the new path.

This alias_generator plugin parses an alias: key in the YAML Front Matter, which is just some YAML in the heading of every Jekyll template file. This per-file configuration allows configuring how the page is rendered.

Installing this custom plugin is easy. Just copy it into ./_plugins/alias_generator.rb and Jekyll will pick up any file ending in ./_plugins/*.rb when compiling.

To setup the "redirect", let's use my example blog post previously at http://johnkary.net/git-add-p-the-most-powerful-git-feature-youre-not-using-yet/. I can use the following alias: key and Jekyll will generate an HTML page at ./git-add-p-the-most-powerful-git-feature-youre-not-using-yet/index.html which redirects to the new post's URL.

layout: post
title: ! "git add -p: The most powerful git feature you're not using yet"
status: publish
type: post
published: true
alias: /git-add-p-the-most-powerful-git-feature-youre-not-using-yet

The plugin then sticks an HTML file at the path specified and anyone encountering that page will be forwarded to the new URL.

<!DOCTYPE html>
<html>
<head>
<link rel="canonical" href="/2012/10/30/git-add-p-the-most-powerful-git-feature-youre-not-using-yet.html">
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<meta http-equiv="refresh" content="0;url=/2012/10/30/git-add-p-the-most-powerful-git-feature-youre-not-using-yet.html">
</head>
</html>

Moving Comments to Disqus

All of my old comments were stored in WordPress. But now I have no WordPress.

Disqus provides comments as a service via a JavaScript widget. They also provide good instructions for abandoning WordPress comments.

I chose the "Manual Import" method because frankly I didn't want to bother installing the plugin. I used my previous wordpress.xml dump from before and uploaded it through their site. The comments imported correctly but at the old WordPress URL paths.

The problem here is, I moved my blog posts from being at the root of my domain (e.g. /my-title-here) to instead live under /blog/my-title-here.

At time of writing, Disqus allow you to address this issue from its Dashboard by instructing its spider to crawl your domain and process any 301 Redirects you have setup. So my .htaccess redirects have caught those and the URLs associated with my old comments are now on their new respective pages.

Well done, Disqus.

Source Code Available

The source code for this site is available on GitHub: https://github.com/johnkary/johnkary.net

That's It!

Overall I'm really happy with this new setup. I feel much more in control of my site, and am much more comfortable writing into a text editor with vim bindings than a WordPress textarea.

I looked at Octopress but felt it did too much for what I wanted to accomplish. I wanted a more micro-approach.

I may also redesign the site at some point in the future, maybe to make it responsive, but right now double-tapping on the content body it's still very readable and focuses only on the content.

If you see any problems with my site, please ping me on Twitter @johnkary

Update January 20, 2014 - This old post from 2012 has again become popular on Hacker News. Since first publishing this article, I've learned that the way I presented editing hunks in the second video, where I show how to edit the line spans at the top of the diff, is both not very efficient and rarely necessary. Hopefully the rest of that video is still useful.

git and GitHub have revolutionized not only how I do development, but how we developers share code.

I have two tools I need to do software development: an editor and version control.

The Pragmatic Programmer advocates you should know your editor inside and out. So why don't you know your version control system just as well?

Patch Mode

If you use git, you've used the command git add. But do you know about git add's "patch mode" using git add -p ?

Patch mode allows you to stage parts of a changed file, instead of the entire file. This allows you to make concise, well-crafted commits that make for an easier to read history.

Since I learned about it a year or so ago, patch mode has taken my git usage to another level. It's one of the most powerful git commands, right up there with rebasing, but much less dangerous!

Screencasts

I have recorded two 10-minute screencasts about how I use git add's patch mode. And I want to share them with you for free.

In the first video I walk you through why patch mode is something you should be using today, then demonstrate its basic features.

In the second video I talk through the dark art of editing hunks. Even some of the most knowledgeable git users I know have struggled with this! To really master patch mode you will want to know how to edit hunks. Never fear "patch does not apply" again!

I hope that after watching you will be armed with a new skill you can start using right away and begin to know git at a deeper level.

Intro to git add patch mode tutorial

One way to edit hunks in git add patch mode

Over the last few years, especially since the release of Lion in July 2011, there is an air of neglect floating amongst the development community that use Apple hardware running OS X as their primary workstation. Many developers feel Apple is not focused on their best interests as developers and are frustrated with new Apple products.

We developers spend much of our time on a laptop or desktop doing research, programming and debugging code, and testing our latest software. Our computers are our lifeblood. They become extensions of ourselves. When something about them changes that hampers our productivity or ability to do work, we get very frustrated. And then we write scathing blog posts and passive-aggressive tweets on Twitter.

My New Air

I recently upgraded from a mid-2008 17" MacBook Pro to a mid-2012 13" MacBook Air. The MacBook Pro I upgraded from was my first Apple machine. I loved Leopard (10.5) and Snow Leopard (10.6) and felt very at home and productive in the 4 years I used them. But the new MacBook Air dropped me right into Lion (10.7).

Like many before me, I bemoaned many of the small changes between Snow Leopard and Lion, mostly because many of them decreased my productivity by acting counter to how I'd been accustomed to it, or software I had previously relied on simply didn't work in Lion.

This was my lifeblood Apple was messing with here. This is how I make my living. I can't have a computer that I feel is working against me or takes away my ability to work efficiently.

Now in the 2 weeks since, I've grown accustomed to a few of Lion's nuances (I've embraced the natural scrolling, for example). But it finally clicked when I saw Apple Q3 financial reports yesterday. In it I finally had statistical evidence to backup the feelings shared in the development community that Apple doesn't prioritize OS X anymore.

Back to the Mac

When Lion was announced, it was unveiled to the mantra of "Back to the Mac", where Apple announced they would be taking some features from iOS and integrating them into their desktops running OS X.

What Apple must have seen at the time was that the barrier to entry for Apple into many consumer households was now very low. As low as a FREE subsidized iPhone from a cell phone carrier. No longer were their bold designs and refined hardware relegated to $1000 computers. General consumers could afford a free or $99 iPhone. And they did. They snatched up iPhones in droves helping make iOS the most prolific Apple software ever.

The Q3 2013 financial results unveiled on July 24, 2012 tell an interesting picture. This graph by MacRumors sums up Apple's current revenue picture and how it really drives home the success of iOS-enabled devices:

Most notably, look at iOS-enabled devices vs OS X-enabled devices. Here's the math:

iOS-Enabled Devices:
iPhone: 46%
iPad: 26%
iPod: 3%
= 75% of company revenue

OS X-Enabled Devices:
CPU Net Sales (MacBooks, iMacs, Mac Pros, etc)
= 14% of company revenue

75% of Apple revenue comes from iOS-enabled devices. 75%! Roughly, for every $1 Apple brings in on MacBook or iMac sales, they bring in 5 times as much in iPhone, iPad and iPod sales. 500%!

As a company, where would you put your smartest engineers? Probably on the software reaching not only the majority of your customers, but likely the software reaching your first-time customers.

How did Apple get here?

iOS-enabled devices have exploded in the last 2 years, and Apple's shift away from developing OS X as its flagship core product have followed suit.

Another graph from MacRumors shows Apple's revenue by financial quarter, where we can see this transition from the start of 2009 to current mid-2012.

Pay specific attention to the blue area (OS X devices) compared to essentially everything above it.

OS X-enabled devices were generating about half as much revenue as iOS-enabled devices in early 2009. But in 3 years since, we've seen a huge shift in where Apple's revenue stream comes from, up to the 500% we saw in the previous graph. We developers should be silly to think Apple's engineering efforts have not followed their products' revenue potential.

Future of development computing

As developers, iOS is not yet at the point where it can fully replace a desktop-based machine for software development. The majority of developers will continue using powerful desktop and laptop machines to do our programming.

Unlike the early days of Apple when the company embraced and found refuge in the hacker mentality, Apple has emerged as a consumer company thanks to the wide reach and accessibility of its iOS products.

As hackers and makers we must realize we are no longer Apple's target market. OS X will continue to cater to the lay-consumer to function more like the iOS device they latched onto as a gateway into the Apple product line, all the while giving developers just enough to continue building web platforms and mobile apps that perpetuate Apple's mobile platform.

But Friday, July 20, 2012 was my last official day as an employee of Information Technology at the University of Kansas. I had worked at KU within the same group for almost 8 years (although the group changed drastically over the time I was a part of it, even as it continues to change.)

I started in late 2004 as a student web developer, was hired as a full-time programmer in 2007, and was promoted into a programming leadership position in 2008. In the 4 years since, I worked tirelessly to mold our group of cowboy coders into a modern software development team.

I dedicated a lot of time to establishing high standards for quality within the group, which the team was able to meet. Every developer in the group, including me, came a long way to where they are now, and I hope that even though I will no longer be part of the group that the standard for quality will rise even higher.

Why are you leaving?

My old group's primary responsibilities had both shifted and greatly expanded within the year prior to my departure as a result of both changes in IT leadership and in initiatives demanded as part of a push for efficiency within the greater University.

These new initiatives left even less time dedicated to development work and instead involved a lot more end-user support for various web development and the shared-web hosting platform we offer. Even then, the development work our team was tasked with (building out a campus-wide Drupal-based content management system) I did not find particularly exciting nor rewarding.

I gave it a chance for 6 months, but my suspicions were confirmed that it was time to move on. There is certainly no shortage in demand for application development, and I have been fortunate enough to have been offered a few new opportunities.

Where are you going?

For the immediate future I will be involved in various freelance/contract projects. I actually have a 9-month contract with the KU's Applied English Center to take their 37-year old database application written in Visual FoxPro and modernize it as a web application built on the Symfony2 framework.

I will also continue my long-time contract work with RapidHire, LLC, where my custom web application plays a part in allowing trucking companies to quickly qualify and hire new over-the-road truck drivers.

It's a strange feeling… for the time being I am actually working in my old cubicle at KU as part of my KU-based contract. I left on Friday, and here it is Monday, and I'm back in the same environment, still surrounded by all of my previous teammates. I hear their conversations about the projects we worked on before I left, and I could even verbally answer some of their questions, but my duties and paycheck now come from an entirely different place, so I must focus on my new project.

Honestly, this still all feels surreal. Derek Sivers sums up my feelings perfectly in his book, "Anything You Want", (and in this post on his blog) where he describes his feelings surrounding selling his company CDBaby, which he grew from being self-employeed to 85 employees and millions in CD sales:

I realized that the bigger learning and growing challenge for me was letting go, not staying on.

My previous group at KU will continue to evolve and grow, and I wish them all great success. They are in good hands, and I could not have asked for a better manager in Matt Garrett, who gave me every opportunity to succeed and embraced almost every suggestion in how we could improve the team's abilities, our development practices and the greater IT organization.

And if I achieved my goals while I was in KU IT, I will be replaceable.

If you are interested in hiring me for contract work or a potential full-time position, I can be reached at john at johnkary dot net.

When the iPad debuted in April 2010 I didn't feel I had a use for it that justified the cost. I mean, "my laptop can do anything," I thought at the time. "Why would I want a crippled computer?" Was the world really ready for the new device Steve demoed?

But two years later, the tablet as a device is now a sustained product (I'm reluctant to call it "matured"), has first-class content available for it, and has carved its own niche in the device landscape. That $650 in my wallet is starting to feel mighty heavy…

My device usage

I currently spend most of my screen-time working from a 17" MacBook Pro: the normal 8-ish hours for my full-time employment, then after hours doing some reading, researching, personal programming projects and open-source work.

The iPad would mostly take over as a device for casual browsing with Instapaper, HackerNews, Twitter and RSS. It would also see the occasional movie or gaming while traveling. A single device for these purposes would allow me to focus on reading, instead of being distracted by email, code, chat and all the clutter that comes with normal work when using my laptop.

Much of this type of usage would happen in the late evening and into the night.

Yet there is one killer app I use on Mac OS that isn't natively available for iOS, but makes working in the evening exponentially more comfortable: F.lux.

F.lux is a little app that runs in the background and automatically adjusts the color temperature of the screen according to the current position of the sun in the sky.

During normal daylight hours the screen is unaffected, resulting in light emitted at about 6500K, or about the same color temperature as sunlight radiating under a clear blue sky. But as the sun sets and the color of the light outside changes, F.lux gradually transitions the color temperature of the screen from 6500K to 2700K, which is closer to that of indoor incandescent tungsten lighting. This color-shift makes the laptop screen appear similar in color to your working environment, making it less jarring and easier on the eyes.

The human brain is very good at determining colors in varying lighting conditions and deciding, "that's blue" or "that's white".

It's so good that you often will not notice a difference in color because all other colors are contextually and equally shifted towards the blue or orange end of the spectrum depending on the light source.

You are likely only aware of a difference when looking at a picture and being removed from the actual experience. If you're familiar with fixing the white balance of a photo, your brain is doing exactly that, all in real time.

This effect is most pronounced when comparing color swatches side-by-side and deciding which one appears more blue or more orange.

Source: http://en.wikipedia.org/wiki/File:Grays.svg

When working in a mixed light source environment, such as an orange incandescent lamp illuminating your living room while you work on your couch with the blue-white glow of your laptop, your brain is actually in a constant tug-of-war.

The brain is trying to create an equilibrium by balancing the colors in the room towards the blue of the laptop, and the laptop light towards the orange of the room. But switching context between looking at your laptop and glancing around the room creates a constant battle for how the brain balances the colors the eye sees. You might better know this tug-of-war as a headache.

F.lux stands to save your headache by correcting this contextual imbalance and tuning your laptop screen to emit light close to that illuminating your working area. Just select from the presets for the different types of lighting: tungsten, halogen, fluorescent, daylight, and a custom setting to set your own color temp.

To demonstrate how beneficial this is, install F.lux and fire up your laptop after the sun has set. Then select "Disable for an hour" from its icon in the menu bar. You will notice how jarring the blueish 6500K screen feels and how it hurts your eyes. It's like hearing a high-pitched screeching noise after listening to the relaxing tunes of, say, Jonn Serrie.

Color temperature's effect on sleep

The F.lux website has an entire page dedicated to research that negatively correlates drowsiness, while positively correlating alertness, in people exposed to light that is close in color temperature to that of a normal computer screen. This is essentially sunlight that makes you feel more alert and less tired.

These findings can be appropriated to say that using your computer or iPad (without F.lux) before attempting to sleep may lead to trouble falling asleep, and/or being less likely to experience the deeper levels of sleep required for to feel fully rested.

What about the iPad

The iPad emits a cool light of about 7000K, which is even slightly more blue than the normal 6500K daylight and even further from the 2700K incandescent light color temperature.

This leads me to believe that using an iPad in the evening hours would only keep me awake, which isn't a desirable side effect. I'd rather be winding down from the day than trying to keep myself awake. Right now I don't have that problem because I use F.lux on my laptop.

But the F.lux authors do not offer a native iOS app. Instead, they offer an unofficial iOS app that can only be installed after jailbreaking the device. This is because there are no open APIs to hook into adjusting the colors of the display.

I would love it if Apple incorporated this type of color shifting technology into its hardware by adding a sensor for light temperature and allowing the screen to adjust. It would work as an optional setting, as the color shifting is greatly undesirable when doing work like photo editing, where the screen's color is of utmost importance.

I think the general public would see this as magical, and could even be marketed as a health benefit—which surely could earn points for Apple in the mass media, who likes to inflate stories about how extended use of technology is bad. Maybe this could even bring the concept of color temperature and its physiological effects more into the mainstream awareness.

Thoughts?

It may seem silly that the absence of a single small application would make me think twice about buying a shiny new gadget. But that's really because of the positive impact it has had on me.

It's one of those things that after you start using it, you can't imagine going back.

Have you noticed this jarring effect when using a laptop in the evening?

Or maybe you already use F.lux. Is it as revolutionary for you as it is for me?

Has anyone installed the F.lux iOS app or found an alternative that makes reading from a screen easier at night?

Most changes will probably not affect your every day use of git, but a few new features should be useful.

git merge learned the --edit option to allow editing the merge commit log message

If your development revolves around feature branches, you probably already branch and merge daily. Depending on your preference, you might preserve merge commits in your log history. If you abide by gitflow or manually specify the --no-ff option when merging, you are preserving the merge commit.

By default, these commits are auto-generated by git and don't tell you much. They list which files had conflicts, if any, and the name of the merged branch. Unless there is a conflict, you are not given the chance to edit the commit message when making the merge. But thanks to the new --edit option, you can now add a custom message when merging.

Using --edit will cause git to perform the merge then drop you into your core.editor application to edit the merge commit message itself before finally writing it to your history.

You could use this functionality to write a quick all-inclusive summary of a feature you are merging and how it might integrate with other pieces of your software. If I'm working on an internal team project, I'll also include any commands my co-developers need to make in lieu of these new changes.

git merge --no-ff --edit

git grep learned --untracked option

If you aren't familiar with the Unix tool grep, it basically allows you to search for text within other blocks of text. That block of text may be a large log file or any other type of output sent to your console.

git grep is similar to normal grep, but provides grep functionality within the context of a git repository. For example, git grep takes into account the files ignored in .gitignore and will not return results found in those files.

With the new --untracked option, git grep can now search through untracked files as well. So if you have untracked files you might have created but not yet added to your project, you can search through them as well.

git grep --untracked "string to find"

git diff learned --function-context option to show the whole function as context that was affected by a change

When displaying a diff, a one-line change is normally shown with 3 lines of context. This means the 3 lines both before and after the change are displayed to help remind you where this change was made.

 (master) ~/Sites/hallcenter_awards/symfony $ git diff
diff --git a/apps/applicant/modules/application/actions/actions.class.php b/apps/applicant/modules/application/actions/actions.class.php
index 5092da4..ec22e6d 100644
--- a/apps/applicant/modules/application/actions/actions.class.php
+++ b/apps/applicant/modules/application/actions/actions.class.php
@@ -21,7 +21,6 @@ class applicationActions extends BaseActions</p>

<pre><code>     $this-&gt;route = $this-&gt;getContext()-&gt;getRouting()-&gt;getCurrentRouteName();
</code></pre>

<ul>
<li><pre><code> $this-&gt;competition = $this-&gt;getCompetitionBySlug($request);
 $this-&gt;forward404Unless($this-&gt;competition-&gt;isPassedOpenDate(), 'Competition not open for Applications yet.');

 $this-&gt;applicant = $this-&gt;getUser()-&gt;getGuardUser();
</code></pre>

<p>

If your change is within a larger function or block of code, like the previous example, 3 lines of context isn't enough to tell exactly where in the function this change was made. git shows us the class declaration as context (seen above as class applicationActions extends BaseActions), but wouldn't it be great if we could see the entirety of the function where the change was made?

The new option --function-context (or its -W short option) attempts to provide you this context, giving you all lines of the function within which your change was made:

 (master) ~/Sites/hallcenter_awards/symfony $ git diff --function-context
diff --git a/apps/applicant/modules/application/actions/actions.class.php b/apps/applicant/modules/application/actions/actions.class.php
index 5092da4..ec22e6d 100644
--- a/apps/applicant/modules/application/actions/actions.class.php
+++ b/apps/applicant/modules/application/actions/actions.class.php
@@ -11,203 +11,202 @@
 class applicationActions extends BaseActions
 {
     /<em>*
      * Executes before every action
      *
      * Sets navigation breadcrumbs
      </em>/
     public function preExecute()
     {
         $request = $this->getRequest();</p>

<pre><code>     $this-&gt;route = $this-&gt;getContext()-&gt;getRouting()-&gt;getCurrentRouteName();
</code></pre>

<ul>
<li><pre><code> $this-&gt;competition = $this-&gt;getCompetitionBySlug($request);
 $this-&gt;forward404Unless($this-&gt;competition-&gt;isPassedOpenDate(), 'Competition not open for Applications yet.');

 $this-&gt;applicant = $this-&gt;getUser()-&gt;getGuardUser();
 $this-&gt;application = $this-&gt;getApplicantCompetitionApplication($this-&gt;applicant, $this-&gt;competition);

 $this-&gt;setSlotBreadcrumbCompetition($this-&gt;competition-&gt;getTitle(), 'application_overview', array('slug' =&gt; $this-&gt;competition-&gt;getSlug()));
 $this-&gt;setSlotBreadcrumbCompetitionPage('Application Form');
</code></pre>

<p> }</p>

<p> /**</p>

<ul>
<li>Displays Application form
*</li>
<li>If Applicant has already started applying, display their info
*</li>
<li>@param sfWebRequest $request
*/
public function executeNew(sfWebRequest $request)
{
 $this->redirectIf($this->applicationExists($this->application), array(
     'sf_route' => 'application_edit',
     'slug' => $this->competition->getSlug(),
 ));
}
// basically continues until the end of the file

I've found this option rather unreliable, at least within a large PHP class. My tests found --function-context often results in displaying almost all of the original file, and git appears ignorant of PHP's function boundaries. The number of context lines before and after a change seem random, and the diff doesn't necessarily always show all lines of the function, either.

The original patch message that introduced this change sheds some light:

This implementation has the same shortcoming as the one in grep, namely that there is no way to explicitly find the end of a function. That means that a few lines of extra context are shown, right up to the next recognized function begins.

So it appears detecting a function's boundaries is difficult for git. It seems in this instance, git never detects a function boundary and gives us the context of the entire file.

Perhaps support for this functionality will improve in future versions.

For now, git diff already supports the --unified= option, or its short version -U, for displaying a larger number of context lines than the default of 3. At this point, I can only recommend using --unified= to display more lines of context in your diffs.

git diff --unified=10

Jobs loved what he saw. Xerox had developed one of the first GUI's, one that would ultimately lay the foundation for the concept of "windows" (small W) and the "desktop" we know in modern operating systems.

Jobs saw the potential of such a user interface, and an ability to capitalize on Xerox's early work, but deliver it at a fraction of the price. He immediately tasked early Apple employee Bill Atkinson with replicating what the two had seen at Xerox.

Atkinson's finished product not only faithfully reproduced what he and Jobs had seen at Xerox, but Atkinson also inadvertently added a revolutionary feature we now take for granted: the ability for windows on screen to overlap.

Biographer Walter Isaacson writes in Steve Jobs:

"Atkins pushed himself to make this trick work because he thought he had seen this capability during his visit to Xerox PARC."

Atkinson told Isaacson:

"Because I didn't know it couldn't be done, I was enabled to do it."

Sometimes we think we know the rules and limitations of an idea. Until someone comes along and breaks those rules, simply because they didn't know such rules existed.

It's amazing what can be done when nobody tells you you can't.

By the end of this article I hope you have a better sense of how you choose what to listen to, and that I introduce you to some new artists to increase your productivity when working.

The most important part for me in finding The Zone is my auditory environment. Where I work varies: the office, at home, 30,000 feet in an airplane, etc. Each environment has its own accompanying distractions: conversations in the next cubicle row, people walking by, my neighbor's rhythmic subwoofer, the barista making a drink order, the impending "Would you like something to drink?" from the flight attendant making her way down the isle.

Music and quality audio equipment have become my great equalizer.

But music alone isn't enough. I can't slip into The Zone just listening to just any music. The type of music that makes me most productive is affected by many factors:

• What am I working on? Find a beat and rhythm to match the task. Is it a long, enduring task like fixing an epic bug? Or am I drudging through mindless work and need something to liven it up?
• Does the task require internal dialogue, like reading or writing? Music with lyrics interferes with my thinking. I process things internally by talking to myself in my head. Someone singing in my ear creates another voice commanding my attention, detracting me from clarity.
• What's the weather like? I seem to prefer different types of music given the weather. A sunny Summer day calls for different music than a blustery Winter day.
• What time is it? If I'm winding down for the day I probably don't want to turn on some 140 bpm Trance. That would just keep me awake. Unless that's exactly what I need.
• What mood am I in? Maybe my morning didn't go so well, or maybe I just pushed a great new feature and am reeling with excitement.

Whatever the case, what I pick to listen to is both an extension of what mood I'm in and what mental state and mood I need to put myself in to be productive in what I'm working on.

Early Morning (5am - 8am) - Low-key

I use my mornings to catch up on Twitter, articles I've saved to Instapaper, or sometimes experiment or test out solutions to problems I've been mulling over. Whatever the case, I don't like much distraction.

The darkness and lack of buzz before sunrise allows me to focus. No one is moving except the birds and squirrels outside. Jason Friend and DHH hit it perfectly in their book "Rework":

Think about it: When do you get most of your work done? If you're like most people, it's at night or early in the morning. It's no coincidence that these are the times when nobody else is around. […] Long stretches of alone time are when you're most productive. When you don't have to mind-shift between various tasks, you get a boatload done.

Jazz and low-key minimalism tunes allow me to get my brain moving and cognitively functioning. It's like morning coffee for my ears. Contemplative, but energizing.

Music:
(Listen) Goldmund
(Listen) KILIMANJARO
Gonzales - Solo Piano
Miles Davis - Kind of Blue

Morning Work (8am - 12pm)

By now I'm in the office and ready to start on the day's work. Check email, put out any fires, start structuring my plan for the day.

Right now, I need music to keep me deep in thought and focused. The office usually has many people running around, or having conversations. My goal is to tune them out.

This music is a bit spacey and droning. The minimalism and calming nature also keeps me level-headed about any boogiemen I might discover in my Inbox.

After filing and sending email, it's usually any new support work or maintenance programming on my current project. Tying up loose ends so I can have a productive rest of the day where I'll write new code, perhaps continuing what I was building on earlier in the morning.

Music:
(Listen) Cliff Martinez - Solaris Soundtrack
(Listen) Jonn Serrie

Afternoon (1pm - 5pm)

The post-lunch time slot can be difficult. It's either a continuation of a productive morning, or an uphill battle to stay focused. Depending on the day, over lunch I either played basketball, watched a video/conference talk I saved via Instapaper, read a programming-related book or caught up with co-workers.

I try to pick music to draw me back into my productive morning state, or kick-start me into The Zone if my morning yield was poor. Caffeine with lunch seems to be a reliable way to get me going in the afternoon and keep me focused. Adding the right music to that allows me to get a lot done.

Music during this time varies so much because it depends how my morning went. I usually kick iTunes into Shuffle and play tracks until something hits the spot.

I'm usually in this mode until the end of the work day.

Music:
Electronic / Trance - DI.fm Trance Channel or Armin van Buuren
Radiohead
Nine Inch Nails
August Burns Red
As Tall As Lions
The Mars Volta
Circa Survive
Pendulum

Night (10pm - 1am) - Drifting

By now, errands have been run, dinner is finished and cleaned up, I've spent time with friends or my girlfriend, and I'm ready to get back to it.

If I had an early morning or a long day, there's usually another window of quality focus during the nighttime hours. There's not much coming through on Twitter, not many people on chat, and nothing of interest on TV. I can focus again.

This is another period of quality isolation time, allowing me to ignore everything else from the comfort of my couch and crank on more code or work on a blog post. (It's 1:30am as I type this.)

I try not to let myself go to sleep if I'm in the middle of coding a feature or fixing a bug. There's time investment required to orient my mind to what I'm doing, like having a virtual machine running, a few terminal sessions going, a few tabs open in NetBeans and a dirty working tree in git.

But I can usually find this weird blissful mental state where I feel half-asleep but can focus intensely on what I'm doing even though I'm really tired. It's like I can ignore anything else on my mind from the day and just sink into a comfortable place and code. I get lost in the music and time stands still.

Music:
(Listen) Godspeed You! Black Emperor
(Listen) Benn Jordan - Pale Blue Dot
(Listen) Sven Weisemann
(Listen) Jonn Serrie
(Listen) Maserati
(Listen) The Flashbulb
(Listen) The Mercury Program

Most changes will probably not affect your every day use of git, but two new additions should prove very useful for many developers:

"git stash" learned an "--include-untracked option"

git stash is one of my favorite git features. If you don't know about git stash yet, it allows you to take all of the changes in your working tree that haven't been committed and temporarily store them, then cleans your working tree.

Basically it tells git, "Hey, I'm not ready to commit all of these changes, but I really need to get back to the way my repository was right after I made my last commit. So save my most recent changes with this given description, so I don't forget what I was working on, and I'll come back to them later."

Prior to git 1.7.7, if you ran git stash save it would only stash changes for files that were already under version control. So if you had created a new file that was part of the feature you were working on and wanted to stash it, you were stuck with an ugly work around of having to add it to the repo using git add, then stashing your changes. When later replaying the stash, those files would still appear in the staging area/index.

But with the new option --include-untracked, or simply -u, git stash can now save your working tree exactly the way you had it when you created the stash.

git stash -u
git stash --include-untracked

"git submodule update" reports any errors at the end instead of halting on error

git submodules allow you to embed and track a foreign repository in your parent repository. They are great when you want to distribute code between multiple projects, but keep the distributed code in its own repository with its own version history.

Prior to git 1.7.7, if you ran git submodule update and there was a problem, git would immediately halt. Halting in the middle of updating might update some of your submodules and leave others unupdated.

But now, git will continue updating all submodules and instead report a list of errors after attempting to update all submodules.

Sagan lays out two over-arching requirements that must be met for the survival of the human species, which he reinforces in various ways throughout the book:

We must venture outside Earth, and ultimately outside the dependency of the Sun, if humans are to survive the end of the Earth

The Sun and Earth have limited lifespans, albeit billions of years. The Sun will eventually turn into a red giant star (the current estimate being in about 5 billion years.) Increasing in size and amount of light energy emitted during this process, it will alter the atmospheric composition of Earth by increasing its surface temperature.

These atmospheric changes will cause a chain reaction in all living organisms on Earth as the atmosphere and oceans begin to boil away. The balance of life that has evolved while the Earth is in its current stage will cease to exist, perhaps giving way to a new quality of life and organisms better suited to tolerate the new conditions, before all atmosphere, water and upper layers of earth are burned away.

Finally, after all molten matter is stripped from the surface, Earth's iron core will be engulfed by the expanding horizon of the red giant star previously known as our Sun. All human creations, trees, grass, dirt, fossil fuels, buildings, gravestones, and all remains of every organism to ever grace the Earth… all planetary matter will have been broken down into other energy, and ultimately re-distributed into surrounding space, to one day be recombined into (anti?) matter or energy somewhere else in the Universe.

Our history as a species and everything every single human has ever worked for will be gone. The only remaining remnants of our existence will be the light and radio waves that were reflected and emitted during our existence on Earth. Anyone observing Earth from billions of light-years away would actually be observing Earth as we currently inhabit it. Wave to them.

That is unless we destroy ourselves first.

We mustn't destroy ourselves

Whether through rash decision by an ideologic or radical peoples, or by means of ignorance or plain selfishness, the human species has the ability to destroy itself. Whether through technological means such as nuclear war, or selfishness and shortsightedness in damaging the delicate life balance under which current organisms have evolved.

The human species, in its development of technology and its ever-increasing rate of advancement, has harnessed power capable of mass destruction. While this power might seem small in scope of the energy harnessed by the perpetual motion of the Universe, it is enough to destroy our entire world. Whether used recklessly by way of a political statement, or pure accident, the technology enabling this power is now a lit fuse that could easily disrupt our existence.

By establishing a human presence on surrounding worlds and stars, we can lower our vulnerability to catastrophe like this and the fragility of the Earth that ultimately cultivated us.

Sagan argues through much of the book that the first step in exploring the stars is sending humans to Mars. But his pessimistic outlook on the state of the human race suggests he believes humans will destroy themselves and their planet before spacefaring nations can foster the finances to accomplish such a feat.

Given the current financial state of the United States in 2011, it would seem increasingly unlikely for such an undertaking to see approval by its governing bodies anytime soon.

At the University of Kansas where I currently do web development work at the time of this writing, we recently had a random student developer from another large University call our help desk and suggest we implement a collapsable content list (an accordion) on our Academic Schools & Departments page (seen below.)

Even though the content is in a list format with headings over each area, I chose not to use an accordion. Why not? Let's consider a few things.

How did the user arrive at this page?

Looking at our Google Analytics numbers, about 99% of visits to the Academics page come from the KU's homepage and primary site pages. A link to "Academics" lives in the top-navigation, so it's very visible. This indicates the user is already browsing around our primary site, where they find general info about the University.

This particular page is also the 3rd-most visited page on KU's main website, only behind the Homepage and A-Z's "E" page (where users go to click on "Email.") So it's seeing a lot of traffic.

How many clicks deep is this page? What other pages might the user have encountered before arriving at this page?

This page is one click deep at all times. A user can access this page with one click from almost any other page on KU's website, so its content should be geared to hit a wide audience with a variety of expectations.

What is the context of this page?

In this case, the Search tab is active when the user first arrives. Users are accustomed to (and often prefer) searching to find the content they want, so we let them Search first. Search provides instant gratification and is a low time investment opposed to reading a long page of text or links.

We also operate under the assumption that after a user finds the information they seek, they stop looking. If search meets their needs, they will not likely encounter our list of Schools & Departments.

So what can we conclude about the context of this page and the user arriving there?

• The user is looking for info about the schools and departments KU offers.
• Since the user was first provided a Search box, they likely already searched for the info they wanted and their needs were not met.
• They may or may not know the name of the department they're looking for and…
  • want to see a hierarchy of schools and departments
  • want to see all options and pick the one that best matches what they're looking for

Now that we've asked given context to the page visitors, let's pick up where most tutorials start: the content being presented.

What type of content is this?

Our very long list is broken down in a hierarchy by School, then Department. If we took into account only the hierarchy and length, a collapsed list would absolutely solve that problem. But let's go beyond content tutorials and dig a bit deeper.

Does the user need to see all content at once?

In this case, yes, they likely do. The user didn't find what they wanted using the search, and they want to see all possible options to find the one that best matches what they're looking for.

Remember, this user might not know what they're looking for because this page is not very deep in the navigation structure. So rather than hide a bunch of content from the user, we give it to them all at once. They can use CTRL + F to search the page content for a specific department, or quickly skim over each school and through the department lists.

Conclusion

These are all questions you should ask yourself before implementing any type of visual effect or design element. You might think you're being clever, but you could just as easily be hurting the usability of your site's content.

Does any of that really affect you? No, really… does it?

NO! And if you think it does, it doesn't. Want proof?

Before reading any news article today, insert "Who cares if…" in front of the headline.

Who cares if some radical pastor wants to burn Korans?
Who cares if huge fires are engulfing a San Francisco neighborhood? (OK, unless you have family in the neighborhood.) Who cares if there's another meaningless evil zombie movie, now in 3D?

Who cares? Not me, and not you.

We're fed this junk news because the news companies NEED something to cover. If they don't cover SOMETHING they're sending out dead air, or blank newspapers, or a news junkie's cryptonite: re-runs.

It's not just national and local news networks and newspapers. It's all over our digital life too.

Look at your RSS Feed (if you even still use one.) Your Twitter feed. Your Facebook feed. How many of those stories do you actually read? How many do you actually care about? You probably don't even have enough time to keep up with it all.

People follow this junk news so they can keep up on current events, or have something to talk about at the water cooler, or chat up acquaintances at awkward dinners and family get togethers. People get a sense of pride from knowing "what's going on in the world!"

Some people get satisfaction relaying the newest news message to someone else. Or being the first to know something among their friends. It actually validates their news obsession and gives them a sense of being "in the know."

We're so bent on crowdsourcing things these days… how about crowdsource your news reading?

Let everyone else keep up on news for you. Let them tell you about it over lunch or a drink. This will not only free you up for more important things, but reaffirm your friend's need to feel important for catching you up on the big story.

So purge your RSS feed. Stop following Tech Crunch, Gizmodo, Engadget and Apple News. 95% of their stories don't matter anyway.

Unfollow all but a subset of your Twitter crew. Focus on the ones putting out meaningful content and not just noise.

Unfriend all those acquaintances and old high school friends on Facebook. Or stop logging in so often, or close your account all together. Take a friend out to lunch if you want to catch up on what they've been doing instead of passively taking in their life.

If something big is happening, let someone else tell you about it. Then ask yourself, who cares? It won't be you.

MediaTemple asserts in a July 16, 2010 blog entry, which as of November 2012 had been removed from their site: "We do not believe that this is an infrastructure issue, but we are still investigating the root cause(s)."

I believe I have evidence to the contrary.

I noticed today my WordPress blogs had been hit by the Redirect Exploit. I followed the fix instructions and came across something fishy.

Before upgrading one of my old blogs from WordPress 2.5 to 2.6, I had made a full copy of the database. So I then had two identical databases up with different names. I switched the database name in my WordPress config and went along with the upgrade. So my WordPress 2.5 database lay dorment, without an active frontend to administer it.

I ran the fix code against my old WP 2.5 database, and found it had been injected with the exploit code, even without an active WordPress install pointing to it.

This is significant because most WordPress exploits are local to the installation itself, and likely do not affect installations on a host's entire network.

This leads me to believe the exploit was carried out as a result of a system-level security breach by obtaining a list of all databases on all compromised accounts, and running an UPDATE query to inject code into every WordPress database.

MediaTemple's head of support and a few sysadmin/security guys were nice enough to give me a call after I put out this tweet to discuss what I had found, and divulge a bit of what they know.

They believe someone possibly obtained a list of database credentials, then used those credentials to scan and inject code. They said they had already enacted some system changes to mitigate some security issues, and were in the process of re-architecting some aspects of the shared system.

I expect we will see more news from the MediaTemple crew about this exploit in the coming weeks.

To prevent further exploits, you should manually change the password of every database account on your (gs) account, and check that your /domains/ directory is not world-readable (chmod 755 750)

July 21, 7:50pm Update: A previous version of this article had info that could be interpreted as users having access to other users accounts on the system. That was not the intended message and the wording has been updated to reflect this change.

September 30, 2010 Update: Since this article, I have needed to remove malicious JavaScript from this blog two more times, each time with a slightly different-looking exploit code. The most recent exploit I discovered yesterday when a user reported Google was reporting my blog as hosting malware. Sadly, this was not limited to just my blog.

MANY domains on my MediaTemple grid-server account were found to be listed as hosting malware by Google. Even domains without WordPress installed. This points to a larger issue where either the server itself is insecure or a web application on the server (WordPress?) is being used as an attack vector to modify files on the filesystem.

2011 Update: I have since left MediaTemple's (gs) hosting and started hosting on a dedicated VPS. Surprise, none of my sites, this blog included, have not been knowingly exploited since.

I'll admit, I'm in that latter group of striving to write "perfect code." I constantly read articles, pour over well-written open-source code (mostly Symfony code) and review conference presentation slides posted by notable developers on SlideShare.

Like every one of us, I've come a long way from where I was when I started writing PHP. (With a long way to go, still, I'm sure.) I remember the days of constantly hitting syntax errors before discovering IDEs and performing search and replace over an entire project to update code duplicated across multiple files. Yeah, it was a nightmare. One I would prefer to not repeat.

So while I'm not going to read a single article about a design pattern and roll it into my next production product, I am going to continue learning from the best developers in the industry and explore the concepts they advocate.

The Redheaded Stepchild: WordPress (and its many siblings)

Marco also claims that even though WordPress is regarded as "unmaintainable" by many in the community, it has plenty of maintainers and many plugin contributors. But making the "quantity over quality" argument here holds no weight.

WordPress has a lot of maintainers because it still relies on concepts and methods that many junior developers think are OK. Meaning, "WordPress developers" haven't become fed up with their code quality enough to push deeper. WordPress' codebase is how beginners write code, so it is familiar to a large number of developers eager to put their mark on something. Many PHP programmers never progress past this stage. And their ignorance is probably bliss.

Given WordPress' popularity, many junior developers view the code under the hood and think, "WordPress is successful. This must be how you write successful PHP code." NO! It's not! Writing code like that is what makes you hate your job!

Marco argues that if your code works and you are releasing software, you are doing it right. But how much did you hate your life when…

1. you had to fix more than one security issue per release cycle
2. you spent hours trying to fix a single bug, only to have squash it, and it broke twenty other things
3. realizing you missed other occurrences of that same bug after fixing it in the previous release
4. the API/software you write third-party code for breaks your plugin with every release (just like WordPress does.)

Having to fix broken code not done well the first time will cost your client more money, and cost you your patience and ambition. Spend time thinking about your next function before you haphazardly write it. Think about its context, where might it be reused in the future, and what parts could be swapped out to make it more reusable and easier to test.

Where do you want to spend your time… Up front researching, reading and writing quality code up front with DRY principles and hopefully at least some unit tests? Or at crunch time cursing your monitor and slamming your head on your desk agonizing over code you wrote 3 months ago?

Do Something! Learn From the Best

When you're writing code or debugging old code and thinking, "There's got to be a better way…!" There is.

Start with the PHP Advent series: 2009, 2008, 2007.

Learn what Dependency Injection is and how you can use it.

Follow some of the most respected PHP developers on Twitter.

And start writing better code.

Google asked for nominations of cities that might be a good fit for the project, so I wrote a recommendation letter for where I live in Lawrence, Kansas:

Dear Google,

Lawrence, Kansas is a quintessential college town and home to the University of Kansas, with total population of about 90,000 most of the year, and about 70,000 when the University is not in session. Lawrence is relatively small geographically, covering only about 28 square miles, and is home to a talented technology and development community. Now the interesting parts.

Lawrence's local cable provider, Sunflower Broadband, holds a bit of a monopoly on cable service in the area, leaving many residents wishing for more options for Internet access other than Sunflower or AT&T DSL/U-verse. Sunflower is also one of the few cable companies in the nation that use bandwidth transfer limits on most of their tiered accounts.

I subscribe to the middle tier account at 7Mbps. Within the last few months, Sunflower upped the bandwidth cap from 15GB to just 25GB. While additional bandwidth is available for purchase, if you transfer over your limit, you are charged per-GB. Google's presumably unmetered-bandwidth fiber network would give residents yet another legitimate high-speed internet option and provide Lawrence's talented development community bandwidth to play with.

The University of Kansas in Lawrence is also already connected to fiber optic lines that run along Interstate 70, through Kansas City out to Manhattan, KS. While I don't know the politics involved, Google could leverage existing infrastructure and run their own lines parallel to existing lines, making the process of installing new lines more cost efficient.

Being home to a leading research institution in the University of Kansas, Lawrence is also home to many well-connected technology advocates and Internet technology pioneers. The Lawrence development community is very active and produces original high-quality software used by people across the world. A successful launch in Lawrence would generate a large buzz in the technology and social media niche, generating word-of-mouth feedback.

The US Department of Energy recently provided a grant to local power company, Westar Energy, providing the electric company funds to install "smart energy grid" meters in the majority of Lawrence homes. These new meters will allow customers to monitor their energy usage online. If Google were to lay fiber-to-the-curb for home use, they could possibly negotiate a business opportunity with Westar Energy as the data-communications partner for this grant project, perhaps elevating some of the implementation costs.

As outlined above, Lawrence is small enough, yet large enough to sustain such a project. Fiber-to-the-curb would be a welcomed option for another Internet provider to alleviate bandwidth caps, and would likely be used by many in the Lawrence tech community to build next-generation hardware and software. Existing infrastructure would lower the barrier for implementation, while strong influence in the national technology community would bring much feedback from Lawrence's progressive technology community.

Thank you for reading,
John Kary
Lawrence, Kansas resident

But using a little app called DavMail, you use any CalDEV-enabled calendar app to manager your Exchange calendar, including Apple's iCal. From their site:

DavMail is a POP/IMAP/SMTP/Caldav/LDAP exchange gateway allowing users to use any mail/calendar client (e.g. Thunderbird with Lightning or Apple iCal) with an Exchange server, even from the internet or behind a firewall through Outlook Web Accessl.

So assuming your company has OWA setup, keep reading.

Running DavMail, you will be able to interface from your local machine to the remote Exchange server with full Calendar integration, full attendee completion and full free/busy support all via iCal and Mail.app. Let's get started.

NOTE Before We Begin…

At time of publication, DavMail is currently at stable version 3.6.3. A very recent bug fix allows DavMail to work with OWA login screens with some customizations (like we use at the University of Kansas) so I will go through how to build DavMail from source. I assume when 3.6.4 is available, this will not be necessary.

Step 0 - Build from source

Use Subversion to export the latest build from the DavMail repository:

svn export http://davmail.svn.sourceforge.net/svnroot/davmail/trunk davmail

Change into the new davmail directory and use Apache Ant, which comes standard in Snow Leopard, to build the installers:

cd davmail
ant

You will end up with binaries or installers for all supported operating systems, but we're only interested in DavMail.app for Mac. Let's copy it to our Applications directory then launch it.

cp -r dist/DavMail.app /Applications
open /Applications/DavMail.app

Step 1 - Setting up DavMail

Open the Preferences menu from the DavMail application drop-down menu. This is where you enter your OWA URL and set the local ports the DavMail daemon will listen on.

The normal format of the OWA URL is https://exchangeServer/owa/ for Exchange 2007, or https://exchangeServer/exchange/ for older versions. Take note of the Caldev and LDAP ports, as we're going to use them to setup iCal. Obviously you want to use HTTPS here to encrypt communication between your machine and the OWA server.

Step 2 - Connect iCal to your Exchange Calendar

Launch iCal and open its Preferences window. Under the Accounts tab, click the + button to add a new calendar account.

Enter the following options:

Account type: CalDAV
User name: Your username WITHOUT the @domain.com
Password:Your password
Server address: http://localhost:1080/principals/users/username@domain.com (where 1080 is your CalDAV port, and username@domain.com is your full username)

Click Create.

iCal will ask if you want to send your login credentials over an unsecured connection because we entered http://localhost… as our CalDEV server address instead of an HTTPS connection. But because we entered our OWA address as HTTPS, we are transmitting our credentials securely. For more info on security and DavMail, look under "Security" in the DavMail FAQ.

Step 3 - Setup attendee looking in LDAP

DavMail has pretty detailed instructions for doing this on its website, so rather than repeat them here, read the official documentation page for setting up attendee completion using LDAP.

Step 4 - And you're done!

You should now have a new Calendar in iCal and be able to create new calendar events and designate attendees right from iCal.

Step 5 - Bonus: Hide DavMail dock icon

You don't really need to interact with DavMail from the dock since it's more of an agent-application. We can hide the DavMail dock icon by editing the Info.plist and adding the LSUIElement key.

From Terminal, open DavMail.app's Info.plist using your favorite editor. In my case, I'll edit the text file using vi:

vi /Applications/DavMail.app/Contents/Info.plist

We now need to add the LSUIElement key and set it to true. Find the area of code around something like <CFBundleIconFile></key><string>tray.icns</string> and add this new key definition after it:

<key>LSUIElement</key>
&lt;true>

Save the Info.plist file and relaunch DavMail. The dock icon will be hidden but will still appear in the menu bar.

How often do you give away your personal information just because somebody asks for it, but not really knowing what they will do with it? You know what I'm talking about. Signing up for that "free" magazine subscription. Giving your phone number to the cashier when buying that new movie. Plugging in your email to receive order status updates about that $1.49 cable you bought from a company you never heard of but saw on SlickDeals. Like you, I've done all of these. But not without thinking twice about it.

As application developers, we have insight on both sides: consumers signing up for online services, and building our own services for others to use. When building services, you're in one of two camps: those that harness their users' info for marketing purposes, or those that don't. Regardless of where you fall, you have a responsibility to be transparent and open about how you intend to use or not use the data you're given. And how you present your site design, sign up forms and privacy policy (if you have one) play an important role in how much info your users will trust you with.

Tell your users in simple terms: Your data is safe with us

I'm going to guess the majority of us building unique web applications aren't in the market of sharing our users' data with a third-party. We just need a way to track them in our user table. But how do our users know this?

Sure, they could read the privacy policy, but nobody wants to wade through legalese. There are a million apps out there I could sign up for. I just want to use the app. Let me in. Check. Done. But when given a long privacy policy, I can't help but think, "…I don't know what I'm agreeing to, but I'll give these guys the benefit of the doubt."

I feel much better if I'm given a friendly note on the sign up form: "We don't share your info, promise!" In fact, I will usually trust a company even more if they come right out and say it, rather than hiding behind a privacy policy checkbox.

Of course, if you're shooting for the "next big thing" status, you'll eventually need a legalese privacy policy. But why not give the full legal version, and a cliff notes version? (Hello, mortgage industry?)

Tell your users what is going to happen, and do it

When presented with a sign-up form, another thought goes through my mind: "Will I need to verify my email address?" Depending on what I think might happen, I will either give up my @hotmail address (spam-friendly) or my more personal @gmail address.

If you tell your users what is going to happen, and actually follow through on it, you automatically build trust. "Hey! They did what they said they would!" If your user needs to confirm their email address after submitting the form, tell them. Set expectations and set yourself up to meet them.

Ask for all required info on your Register form

If you really, really need a user's address and phone number to start an account, make it obvious. Give a little red * or REQUIRED next to your required fields.

The absolute worst thing a company could do is present a simple Register form (Name, Email, Password), require e-mail verification, then after verifying the address with a URL, require the user to give up their Mailing Address, Phone Number and Birthday before fully activating their account.

Not only will you likely increase the number of "@*\$# you" occurrences in your database, but you are asking users to jump through yet another hoop to use your site. All fields should be on one form with required fields noted, so the user can quickly see which info is required, and if they want to give it to you.

Make the barrier for entry as low as possible. Don't get roped into the backwards old-media ways of thinking! It is not the user's privilege to use your site: it is YOUR privilege that they have chosen to use your site. Without your users, your site doesn't exist.

Don't create more work for the user

If the user clicks Submit and has to correct a few fields, you're not doing your job as a data filterer or a form architect. And you're certainly not building trust. Even if the user enters the wrong info and has to re-complete some fields, they WILL BLAME YOU for not making it more obvious.

This means setting proper labels for your form fields, using color or size to denote required fields and not overwhelming the user. Don't give them a chance to become distracted. You want them to complete your form the first time through instead of making 3 round trips to the server. This not only slows them down from jumping in and using your application, but detracts from the confidence and trust the user has in your application's ability to work correctly and in the way they intend.

User Testing and Production Analysis

A lot of the pitfalls discussed here can be prevented with proper user testing and good feedback questions asked of your test users. Don't just focus on objective data: "How quickly did the user complete the form?" Pull questions on the emotions the user felt when first looking at the site, and after completing registration. "On a scale of 1-5, how much do you trust this company to abide by what they say? To store your data? To perform how you expect?"

The flip side is gathering statistical analysis of your production data after launch. If the bounce rate for your Registration form is high, something is scaring people away. Does your website appear friendly? Is your form well-laid out? Can a user quickly skim through it? Do you treat your users like a humans or spam bots? (Is your CAPTCHA harder than Contra?)

Usage statistics won't tell the full story. What is your clean record to dirty record ratio? Are you collecting real data or are users entering gibberish? Re-evaluate. Do you absolutely need the info you're requesting?

Further Reading

The following are must-reads for architecting web forms and general web content with a slant towards usability and building users' trust.

• Web Form Design Patters: Sign-Up Forms - Smashing Magazine
• Don't Make Me Think - Steve Krug
• Jakob Nielsen's Alertbox: Current Issues in Web Usability
• Clear And Effective Communication In Web Design - Smashing Magazine
• 12 Excellent Examples of "Lazy Registration" - Web Jackalope

Feel free to leave additional reading recommendations or other ideas in the comments. What methods do you go through when laying out your web forms? Is your development front-end test-heavy, or more reliant on A/B results in production?

I will be following the 1.2 to 1.3/1.4 Upgrade Instructions below, so you should read over them before we get started.

Step 0: Create a new branch in your version control checkout for this new version

If you're not using version control, now might be a good time to learn. I am using Subversion for this specific project, but suggest you look at git. The rest of this post will give examples using Subversion for keeping your project up to date.

Let's create a new branch from our trunk to handle upgrading to 1.3.

svn copy http://svn.example.com/repo/trunk http://svn.example.com/repo/branches/symfony1_3 -m "Creating branch of trunk to upgrade to Symfony 1.3."

Checkout your new branch to your development environment and let's get started.

svn co http://svn.example.com/repo/branches/symfony1_3 ~/Sites/myproject/symfony1_3
cd !$

Step 1: Check for 1.3 versions of your plug-ins

Many core plugins were recently updated for 1.3, so check the plugin repository for a 1.3 branch. (Some plugins may work without modification, but this is a case-by-case basis.)

Your plugins may not be as well maintained. Depending on how critical your plugin functionality, you may not be able to continue upgrading at this time.

In my case, I was only using the sfFormExtraPlugin plugin to add some a multi-choice options to my forms.

Step 2: Update Symfony core files to 1.3

Our Symfony core files live in symfony1_3/lib/vendor/symfony. I had manually installed the core files from a ZIP archive when I started developing the project, so they were not yet using svn:externals.

We will remove the old 1.2 files and install the core files from the central 1.3 repository branch. By referencing the 1.3 branch, every time we execute svn up, our Symfony core libraries are updated to the newest version. This may or may not be the best solution in your case. If you do not want your core libraries automatically updated, reference one of Symfony's RELEASE tags.

Since the application I developed is not a critical application, I chose to live on the edge and reference the 1.3 branch.

svn rm lib/vendor/symfony
svn propedit svn:externals lib/vendor</p>

<p>symfony http://svn.symfony-project.com/branches/1.3/</p>

<p>svn commit . -m "Remove Symfony 1.2 core libraries. Install Symfony 1.3 core libraries via svn:externals."
svn up

Step 3: Get newest plugin versions

You can try to update your plugins via the traditional php symfony plugin:upgrade yourPluginNameHere, or check the plugin repository for a 1.3 branch or 1.3-compatible development in the trunk.

sfWidgetFormPlugin has a 1.3 branch, so again, let's remove the old plugin and link up the new one. My previous version was installed via the traditional php symfony plugin:install sfFormExtraPlugin command, so it should first be uninstalled, deleted, then linked up with an svn:externals reference.

php symfony plugin:uninstall sfFormExtraPlugin
svn rm plugins/sfFormExtraPlugin
svn propedit svn:externals plugins</p>

<p>sfFormExtraPlugin http://svn.symfony-project.com/plugins/sfFormExtraPlugin/branches/1.3/</p>

<p>svn commit . -m "Update sfFormExtraPlugin to 1.3-compatible branch."
svn up

Repeat this process for all of your plugins that have new versions available.

Step 4: Upgrading your project code to 1.3

Running this command will automatically upgrade your code base to Symfony 1.3.

php symfony project:upgrade1.3
svn commit . -m "Upgraded project to Symfony 1.3."

Doctrine 1.0 to Doctrine 1.2

Assuming you are using Doctrine as your ORM, Symfony 1.3/1.4 also upgrades Doctrine from 1.0 to 1.2. You will need to manually check your code against the upgrade documentation for Doctrine 1.0 to 1.1 Upgrade and Doctrine 1.1 to 1.2 Upgrade.

In my project, I was using a few non-static methods that had been changed to static methods in Doctrine 1.2.

Propel 1.3 to Propel 1.4

Propel ORM has also been updated to Propel 1.4 in Symfony 1.3/1.4. Check Propel's Documentation for any new changes.

Step 5: Rebuild your models/forms/filters and clear the cache

With all our code updated, it's time to run the gauntlet of updating our forms, filters and models.

php symfony doctrine:build --all-classes php symfony cache:clear

Step 6: Test your site in the browser

Load up the dev environment for your application in your browser and cross your fingers :) Check the Web Debug bar and see that you're running on Symfony 1.3.0 and the version of your ORM.

Assuming everything works, commit back to SVN.

svn commit . -m "Fully upgraded to Symfony 1.3."

Step 7: Check for deprecated code using project:validate

Now that our project is up to date with Symfony 1.3, we need to prepare to move it to Symfony 1.4. Fabien recently added an amazingly helpful command project:validate that checks your code for use of deprecated code that has been removed for Symfony 1.4. Running the command against your project gives a list of deprecated classes in use and their location.

php symfony project:validate

Since I started writing this project on Symfony 1.2.8, and this is a relatively straightforward project, there is not much to change. If you started from 1.0 or 1.1, there is undoubtedly a lot more code to change.

Most of the deprecated code above appears in the /base/ directory of the Doctrine-generated code, which will be fixed when we regenerate our models/forms/filters.

Fixing Deprecated Code

sfWidgetFormDoctrineChoiceMany

project:validate found one use of sfWidgetFormDoctrineChoiceMany in EventForm.class.php. Since I redefined my Event form in the EventForm::configure() method, I need to find the fix for using this deprecated class.

After checking the Deprecated Code page for 1.3, the *Many() form widgets have been deprecated. sfWidgetFormDoctrineChoiceMany has been consolidated into the sfWidgetFormDoctrineChoice class by adding the option array('multiple' => true) to denote a multi-select.

Symfony 1.2:

$this->setWidgets(array(
    'keywords_list' => new sfWidgetFormDoctrineChoiceMany(array(
        'model' => 'Keyword',
        'renderer_class' => 'sfWidgetFormSelectDoubleList',
    )),
));

Symfony 1.3:

$this->setWidgets(array(
    'keywords_list' => new sfWidgetFormDoctrineChoice(array(
        'model' => 'Keyword',
        'renderer_class' => 'sfWidgetFormSelectDoubleList',
        'multiple' => true,
    )),
));

sfValidatorDoctrineChoiceMany

Since I was using a deprecated form widget, I also double-checked the widget's validator for the same field. [sfValidatorDoctrineChoiceMany][] has not been deprecated. Instead, it calls its parent configure() method in its parent class sfValidatorDoctrineChoice, and sets the option multiple to true.

Sounds very similar to the explanation above, doesn't it? Why the developers chose to deprecate the widget and not the validator, I'm not sure. Regardless, I chose to change sfValidatorDoctrineChoiceMany just to stay consistent with my form widget.

Symfony 1.2:

$this->setValidators(array(
    'keywords_list' => new sfValidatorDoctrineChoiceMany(array(
        'model' => 'Keyword',
        'required' => false,
    )),
));

Symfony 1.3:

$this->setValidators(array(
    'keywords_list' => new sfValidatorDoctrineChoice(array(
        'model' => 'Keyword',
        'required' => false,
        'multiple' => true,
    )),
));

sfCompat10Plugin

sfCompat10Plugin is also deprecated in Symfonry 1.4, so we must remove all references to it. In my case, I had excluded it from loading in myproject/config/ProjectConfiguration.class.php.

Symfony 1.2:

$this->enableAllPluginsExcept(array('sfPropelPlugin', 'sfCompat10Plugin'));

Symfony 1.3:

$this->enableAllPluginsExcept(array('sfPropelPlugin'));

At this point, we have removed all deprecated code and are ready to upgrade our project to Symfony 1.4! Commit our latest changes.

svn commit . -m "Removed deprecated code for Symfony 1.4."

Step 8: Upgrade to Symfony 1.4

Once again, you should check if your plugins have Symfony 1.4 versions, and update their svn:external references to their 1.4 branches.

If you plugin does not have a 1.4 version, the 1.3 version may work without modification. If the project:validate task does not return any deprecated code used in plugins, you may be clear.

svn propedit svn:externals</p>

<p>plugins #Update your repo URLs to their 1.4 branches</p>

<p>svn up

With our Symfony core libraries referenced via svn:externals, we also need to update that reference to the Symfony 1.4 branch.

svn propedit svn:externals lib/vendor</p>

<p>symfony http://svn.symfony-project.com/branches/1.4/</p>

<p>svn up

Now regenerate our forms, filters and models.

php symfony doctrine:build --all-classes
php symfony cache:clear

And assuming everything works, commit everything back to our development branch!

svn commit . -m "Upgraded to Symfony 1.4. Whew!"

Congratulations! You are now working with the latest version of Symfony 1.4!

Well that was rather painless! You may or may not agree depending on how much legacy code you are maintaining :)

Congratulations to Fabien Potencier, Fabien Lange, Jonathan Wage, Kris Wallsmith and everyone who submitted patches to make this release possible!

With your new Symfony 1.4 install, why not start learning some of the more advanced tutorials in the Symfony 2009 Advent Calendar?

I've been having a battery/power issue with my MacBook Pro recently where it will turn off if the battery power has much less than 0:10 minutes left. You know, after the "You are running on Reserve Power" message comes up, but before it fully drains and (normally) puts itself to sleep using SafeSleep.

So what do these power issues have to do with Evernote?

I experienced at least one said power failure while working on a note from Evernote's Mac Desktop client. From our days of using Windows, we all know power failures without proper shutdown lead to corrupted files on the hard drive. And that's exactly what happened here: a corrupted notes database file.

After launching Evernote, I would get an error saying, The file "Evernote.sql" could not be opened. Assuming you have recently sync'd your notes so that everything is on the Evernote server, here's how to fix it:

1. Quit Evernote. Force quit if you have to.
2. Open Terminal.
3. Issue this command to remove your Evernote application settings and notes database directory:

rm -rd ~/Library/Application\ Support/Evernote

Now re-launch Evernote. Your notes database and all settings will be downloaded from the Evernote server.

Easier fix than I anticipated. Here are the messages from my Console, in case it helps you find this page via Google.

11/22/09 10:10:20 PM Evernote[301] Client name: Evernote Mac/62233 (en-US); MacOS/10.5.8;
11/22/09 10:10:21 PM Evernote[301] WebKit runtime: 531.21.8
11/22/09 10:10:21 PM Evernote[301] WebCore runtime: 531.21.8
11/22/09 10:10:21 PM Evernote[301] Safari clipper plugin version is 62229
11/22/09 10:10:22 PM Evernote[301] NSInternalInconsistencyException thrown inside CoreData.
11/22/09 10:10:22 PM Evernote[301] NSInternalInconsistencyException thrown inside CoreData.
11/22/09 10:10:22 PM Evernote[301] NSInternalInconsistencyException thrown inside CoreData.
11/22/09 10:10:22 PM Evernote[301] I/O error for database at /Users/johnkary/Library/Application Support/Evernote/data/51783/Evernote.sql.  SQLite error code:1, 'SQL logic error or missing database'
11/22/09 10:10:22 PM Evernote[301] I/O error for database at /Users/johnkary/Library/Application Support/Evernote/data/51783/Evernote.sql.  SQLite error code:1, 'SQL logic error or missing database'
11/22/09 10:10:22 PM Evernote[301] NSInternalInconsistencyException thrown inside CoreData.
11/22/09 10:10:22 PM Evernote[301] NSInternalInconsistencyException thrown inside CoreData.
11/22/09 10:10:22 PM Evernote[301] NSInternalInconsistencyException thrown inside CoreData.
11/22/09 10:10:22 PM Evernote[301] Presenting error: Error Domain=NSCocoaErrorDomain Code=256 UserInfo=0x1543b310 "The file ?Evernote.sql? could not be opened."
11/22/09 10:10:22 PM Evernote[301] Presenting error: Error Domain=NSCocoaErrorDomain Code=256 UserInfo=0x1543b310 "The file ?Evernote.sql? could not be opened."
11/22/09 10:10:22 PM Evernote[301]   user info: {
NSFilePath = "/Users/johnkary/Library/Application Support/Evernote/data/51783/Evernote.sql";
NSLocalizedDescription = "The file \U201cEvernote.sql\U201d could not be opened.";
NSStackTraceKey = "0x37419a  0x92289e3b  0x917a1f1a  0x917a45ab  0x9179d24c  0x91795d20  0x91795a27  0x91794aba  0x9179208b  0x962d8338  0x9617a6cb  0x962d7863  0x962d78e9  0x96406e5e  0x961edafc  0x9478d91d  0x9478d308  0x9478d3d8  0x939ae88e  0x9470e8f5  0x9470eaa8  0x9505f2ac  0x9505f0c5  0x9505ef39  0x95dd86d5  0x95dd7f88  0x95dd0f9f  0x95d9e1d8  0x2aea  0x2";
NSUnderlyingError = Error Domain=NSCocoaErrorDomain Code=256 UserInfo=0x154dfbc0 "The file ?Evernote.sql? could not be opened.";
NSUnderlyingException = I/O error for database at /Users/johnkary/Library/Application Support/Evernote/data/51783/Evernote.sql.  SQLite error code:1, 'SQL logic error or missing database';
}