Garrett St. John

I am a web developer and partner at Bold. This is where I share my thoughts, discoveries and other random bits.

A Great Method for Modeling Git Branches

February 16, 2011

Link: A successful Git branching model

This is probably one of the best tutorials I’ve seen on how to run a successful Git repository. Bravo!

Modeling Our Business Before Taking the Plunge

February 9, 2011

Link: The Magic Spreadsheet or Let’s Talk About Money

Noah Stokes shares on his blog about how we modeled Bold on “paper” before jumping in together. Planning is a very important exercise in business that I feel like a lot of creative-types are more than willing to overlook. Hopefully you find it to be just as helpful for your business.

Extracting Attachments From Emails With PHP

February 8, 2011

Yesterday I wrote about how to read emails with PHP, but I want to dig a bit deeper and discuss another part of the My Slow Low project that needed tackling: Extracting attachments from emails with PHP. For the purposes of this post, I will be specifically discussing file attachments, not HTML inline attached files.

This project was executed with CodeIgniter 2.0 so there are a few libraries and helper functions used that will not be available in other frameworks or with straight PHP, however, their functionality can be pretty easily replicated.

function email_pull() {
	// load the Email_reader library from previous post
	$this->load->library('email_reader');

	// load the meals_model to store meal information
	$this->load->model('meals_model');

	// this method is run on a cronjob and should process all emails in the inbox
	while (1) {
		// get an email
		$email = $this->email_reader->get();

		// if there are no emails, jump out
		if (count($email) <= 0) {
			break;
		}

		$attachments = array();
		// check for attachments
		if (isset($email['structure']->parts) && count($email['structure']->parts)) {
			// loop through all attachments
			for ($i = 0; $i < count($email['structure']->parts); $i++) {
				// set up an empty attachment
				$attachments[$i] = array(
					'is_attachment' => FALSE,
					'filename'      => '',
					'name'          => '',
					'attachment'    => ''
				);

				// if this attachment has idfparameters, then proceed
				if ($email['structure']->parts[$i]->ifdparameters) {
					foreach ($email['structure']->parts[$i]->dparameters as $object) {
						// if this attachment is a file, mark the attachment and filename
						if (strtolower($object->attribute) == 'filename') {
							$attachments[$i]['is_attachment'] = TRUE;
							$attachments[$i]['filename']      = $object->value;
						}
					}
				}

				// if this attachment has ifparameters, then proceed as above
				if ($email['structure']->parts[$i]->ifparameters) {
					foreach ($email['structure']->parts[$i]->parameters as $object) {
						if (strtolower($object->attribute) == 'name') {
							$attachments[$i]['is_attachment'] = TRUE;
							$attachments[$i]['name']          = $object->value;
						}
					}
				}

				// if we found a valid attachment for this 'part' of the email, process the attachment
				if ($attachments[$i]['is_attachment']) {
					// get the content of the attachment
					$attachments[$i]['attachment'] = imap_fetchbody($this->email_reader->conn, $email['index'], $i+1);

					// check if this is base64 encoding
					if ($email['structure']->parts[$i]->encoding == 3) { // 3 = BASE64
						$attachments[$i]['attachment'] = base64_decode($attachments[$i]['attachment']);
					}
					// otherwise, check if this is "quoted-printable" format
					elseif ($email['structure']->parts[$i]->encoding == 4) { // 4 = QUOTED-PRINTABLE
						$attachments[$i]['attachment'] = quoted_printable_decode($attachments[$i]['attachment']);
					}
				}
			}
		}

		// for My Slow Low, check if I found an image attachment
		$found_img = FALSE;
		foreach ($attachments as $a) {
			if ($a['is_attachment'] == 1) {
				// get information on the file
				$finfo = pathinfo($a['filename']);

				// check if the file is a jpg, png, or gif
				if (preg_match('/(jpg|gif|png)/i', $finfo['extension'], $n)) {
					$found_img = TRUE;
					// process the image (save, resize, crop, etc.)
					$fname = $this->_process_img($a['attachment'], $n[1]);

					break;
				}
			}
		}

		// if there was no image, move the email to the Rejected folder on the server
		if ( ! $found_img) {
			$this->email_reader->move($email['index'], 'INBOX.Rejected');
			continue;
		}

		// get content from the email that I want to store
		$addr   = $email['header']->from[0]->mailbox."@".$email['header']->from[0]->host;
		$sender = $email['header']->from[0]->mailbox;
		$text   = ( ! empty($email['header']->subject) ? $email['header']->subject : '');

		// move the email to Processed folder on the server
		$this->email_reader->move($email['index'], 'INBOX.Processed');

		// add the data to the database
		$this->meals_model->add(array(
			'username'    => $sender,
			'email'       => $addr,
			'photo'       => $fname,
			'description' => ($text == '' ? NULL : $text)
		));

		// don't slam the server
		sleep(1);
	}

	// close the connection to the IMAP server
	$this->email_reader->close();
}

I tried to comment the code above as best as possible to convey what it is I am doing with the information. This code is a method within a Controller, but I’ve only included the attachment extraction method of the class for this post.

For more information and to credit those I’ve learned from, please check out David Walsh’s post on Retriev[ing] Your Gmail Emails with PHP and IMAP as well as Chris Hope’s post on Extracting attachments from an email message using PHP IMAP functions.

Reading Emails with PHP

February 7, 2011

Last week I got to do a fun little side project which we called My Slow Low. The site is a simple photo collection of slow/low carb meals for those that are out of ideas on what to eat, but want to stick to their diet. While conceptualizing how we would build the site, the idea of emailing in photos came up. We didn’t want the hassle of account management and were trying to go for a more “mobile capable” option. This was my first time coding for IMAP with PHP and I figured some others could use a jump start from what I’ve learned.

PHP already has a nice IMAP extension which needs to be installed and enabled before going further. The core functionality is all there, but the specifics on how to use it aren’t necessarily all that clear.

Here’s a PHP class I put together to do some basic operations on an IMAP Inbox. It’s a bit tailored to this project, but could be easily revised to fit other needs or extended to be more full featured.

<?php

class Email_reader {

	// imap server connection
	public $conn;

	// inbox storage and inbox message count
	private $inbox;
	private $msg_cnt;

	// email login credentials
	private $server = 'yourserver.com';
	private $user   = 'email@yourserver.com';
	private $pass   = 'yourpassword';
	private $port   = 143; // adjust according to server settings

	// connect to the server and get the inbox emails
	function __construct() {
		$this->connect();
		$this->inbox();
	}

	// close the server connection
	function close() {
		$this->inbox = array();
		$this->msg_cnt = 0;

		imap_close($this->conn);
	}

	// open the server connection
	// the imap_open function parameters will need to be changed for the particular server
	// these are laid out to connect to a Dreamhost IMAP server
	function connect() {
		$this->conn = imap_open('{'.$this->server.'/notls}', $this->user, $this->pass);
	}

	// move the message to a new folder
	function move($msg_index, $folder='INBOX.Processed') {
		// move on server
		imap_mail_move($this->conn, $msg_index, $folder);
		imap_expunge($this->conn);

		// re-read the inbox
		$this->inbox();
	}

	// get a specific message (1 = first email, 2 = second email, etc.)
	function get($msg_index=NULL) {
		if (count($this->inbox) <= 0) {
			return array();
		}
		elseif ( ! is_null($msg_index) && isset($this->inbox[$msg_index])) {
			return $this->inbox[$msg_index];
		}

		return $this->inbox[0];
	}

	// read the inbox
	function inbox() {
		$this->msg_cnt = imap_num_msg($this->conn);

		$in = array();
		for($i = 1; $i <= $this->msg_cnt; $i++) {
			$in[] = array(
				'index'     => $i,
				'header'    => imap_headerinfo($this->conn, $i),
				'body'      => imap_body($this->conn, $i),
				'structure' => imap_fetchstructure($this->conn, $i)
			);
		}

		$this->inbox = $in;
	}

}

?>

A fair amount of this is self-explanatory or commented inline, but I will go over the inbox() method because it is the core functionality. The IMAP inbox is much like an array with a numbered key starting at 1. In the inbox() method, I store that index so that the email can be moved, deleted, or read again later.

Next, the header is stored with the function imap_headerinfo(). This pulls down an object from the server containing information like the Subject, From: address, To: address, and text encoding type.

Using imap_body(), the body text of the email is retrieved. What’s returned isn’t overly clean as it’s just the raw body with boundaries included (see: multipart messages). If the received email is in HTML, there will be a plain text and HTML version included. It’s certainly possible to parse through this data like any email client does, but it’s definitely a little bit messy.

Lastly, ‘structure’ is retrieved with the imap_fetchstructure() function. This is very important if you are trying to access attachments as I was with My Slow Low. In my next post, I’ll go further into the details of saving an attachment from an email and share some more about how I implemented the email processor for My Slow Low.

Update: I’ve posted the follow-up to this article on extracting email attachments with PHP.

Bettween

February 4, 2011

Link: Bettween | Easily Track and Share Twitter Conversations

“Dang it, I know I talked to @sam_h about that a year ago, but what were the details?” This question led me to Bettween. Easily retrieve conversations you’ve had on Twitter, or filter down to conversations between you and another user. Remember that it was in the last 3 months? You can filter conversations that way too.

Slow / Low Carb Dieting

February 1, 2011

Link: My Slow Low

The resident ideas guy at Bold, Noah Stokes, has been doing the Four Hour Body “slow carbs” diet for about a month now. Today he admitted that the choices seem limited on what to eat and was interested in what others are eating. A few hours later, we present you with My Slow Low.

Advanced Regular Expressions

Link: Advanced Regular Expression Tips and Techniques

For those of you that don’t geek out about Regular Expressions like I do, here’s a nice article from Nettuts on advanced Regex techniques.

Becoming an Early Riser – Week 1

January 31, 2011

With the first week of being an early riser under my belt, I thought it appropriate to write about how it went. All-in-all things went well and I really enjoyed my mornings. There were, however, some things I struggled with throughout the week.

Fighting the Urge to Stay Up Late

Fighting the urge to stay up late was really tough. Something in my body kicks on about 9PM and no matter how tired I am throughout the day, I have a second boost of energy. It was very difficult for me on most days to force myself to get into bed and just read. Once I did committed, my body gave in and sleepiness followed within 15 minutes.

Feeling Like I Have Less “Me” Time

Because that alone time at night has become such a part of my life, I found myself feeling like I didn’t have enough time in the day to do the things I wanted to do. In reality, I was awake the same amount of time (or even more) as before my shift in schedule. I started skipping a few TV shows I have always watched in the past (likely a good thing) and shifted a few other tasks to the morning. I’m looking to incorporate exercise as well which should help me to feel like I’m accomplishing more as well as helping my overall energy level.

One habit I did fall into was working longer hours. No good. There were a few days I started working at 6AM and worked until 6PM. My goal was never to work longer, only to be more effective in those hours which I work. I’ve either got to fill my morning hours with non-work related activities or knock off work earlier. Currently, I’m leaning towards the latter.

Early Morning Darkness

In the winter, it’s dark in the morning and I feel like it makes getting up way more difficult. Stumbling into the bathroom to wake up with a shower is a struggle. This is where Steve Pavlina’s tips for waking up really come into play. Practice makes perfect and it’s getting easier each day.

Early Mornings on the Weekend

This last weekend was the first under my new schedule and it was fairly difficult. I made some plans early in the morning on Saturday which made it easier to get up, but Sunday was a struggle. I knew in my mind there was no reason to be awake. If I was up, it was going to be to watch TV because I had no desire to do chores or side projects. I ended up sleeping in until about 9:30AM which was a decent compromise. I was still tired at about 10PM last night which didn’t screw up my sleep cycle too badly.

I think going forward I need to come up with a solid plan for my weekend mornings or I’ll fall into the same issue again. Even though it feels like the natural thing to do at that hour, I don’t want to work on the weekends.

The Plan

All-in-all I’ve really liked this new schedule. I plan to stick to it and hope that this first week wasn’t possible just because of the excitement behind proving I can do something new. I’ll continue to practice getting up so that I get better at ignoring the voice in my head telling me to stay in bed. With a few refinements, I really think this is something I could find good success in by changing permanently.

Adding an Empty Directory to a Git repository

January 28, 2011

Link: How to add empty directory to Git repo using .gitignore

I was in the process of checking in a CodeIgniter project today when I realized that the ‘cache’ and ‘logs’ folders were being ignored. My .gitignore was set to ignore the content of those directories, but I did want the actual directories included. After a bit of Googling, I found that empty directories are automatically ignored by Git. The above link shows how to force the inclusion of empty directories without including their contents.

Counting the Costs of the Mac App Store

Panic announced yesterday in a blog post that they have “half released” their latest update to Transmit. It seems the update was submitted to Apple for review, but has spent over 2 weeks going through the process. Meanwhile, under pressure from the Support Team, they have gone ahead and pushed the release to direct buyers. It made me think…is the added value of a Mac App Store really worth the dependence on a third party (even if that third party is Apple)?

At launch, I really liked the idea of the Mac App Store. I felt that is served as a nice, centralized area where I could go to find apps, conveniently checkout with a provider I trust, and come back later for updates in one single place. However, it didn’t take long for the critiques to start rolling in and most of them made a lot of sense to me. But what about Panic’s latest issue? What if this release was a major bug in a core part of their software? Would users be left waiting for weeks on end?

With the Mac App Store being so new it’s no real surprise there are some glitches in the system. Here are a few issues I see that make taking the plunge a tough call:

Apps For iOS and OS X Are Different Beasts

The biggest fault I see with the Mac App Store is that it’s hard to justify the need for it. When the iPhone was first released the App Store made a lot of sense because we were talking about a smartphone with an Internet connection that could be exploited. I would have been somewhat surprised if a carrier was even willing to carry the iPhone without some controls. Apple’s review process minimizes the junk (right?) and clears out the malicious. Everyone is happy because cell phones aren’t being treated as P2P nodes, taking unexpected photos, or constantly tracking our GPS coordinates.

On the other hand, the Mac App Store is aimed at my computer. I don’t need anyone monitoring what I’m allowed to install. If a developer puts out a malicious (or even just a junky) application, the word is going to get out that it’s trash and shouldn’t be installed. If I don’t like it, I uninstall it. There’s really no need for a review process.

Lack of Control of the Experience and Process

As a developer, I value the experience that my user’s get very highly. It’s important to me (and I know even more to some of you) that my work is presented in a certain manner with a specific flow. While the Mac App Store doesn’t control the in-app experience, it does strongly effect the buying experience which is just as important (especially if you have mouths to feed at home). I will admit that the simplicity of the Mac App Store is great, but there are costs involved with surrendering this portion up to Apple.

Lag Time in Pushing Updates

As demonstrated with Transmit, it can be tough to push an important update through the system. This plays into the loss of control from my previous point. We all strive to create bug-free code, but also know it’s impossible. You better believe that when I find something, though, I’m going to fix it now and get it out fast. Not so with the Mac App Store.

The other cause for concern I see here is for those apps that are in early development, like Sparrow. I’ve been using it for some time now and it’s become very stable and offers a great feature set, but earlier on it was being updated quite often. Does this mean that Beta apps aren’t meant for the Mac App Store? Seems a shame to require apps to be fully matured before they “work” in the Mac App Store. Since apps I already have installed are recognized, wouldn’t it be nice if Apple helped Beta users transition into paying customers for v1.0? No deal.

The Cost of Convenience

I understand that not everyone is a web developer and some may not be interested in building and operating their own e-commerce website to sell their software. For this reason the Mac App Store is hugely convenient. But is it worth a 70/30 profit share with Apple? Obviously some would say yes because of the added exposure, but not everyone will be making $1 million in 20 days. With services like Quixly, digital sales and distribution aren’t an overly difficult thing to coordinate. For me, I’m taking home the extra 27% and working with PayPal (yuck) or Authorize.net.

So What Then?

All-in-all I do still think the Mac App Store is a great idea and will likely become more and more of a cash cow for Apple. I like the functionality and potential for what it could be, but I think by surrendering the experience and process to a third party, developers are left at the whim of their provider. Seems like something that definitely calls for counting the cost.