Charles Hooper

"I think and think for months and years. Ninety-nine times, the conclusion is false. The hundredth time I am right." -- Albert Einstein

I think Einstein was onto the equal-odds rule.

For the unfamiliar, I wrote an article about a week and a half ago titled How I Made Money Spamming Twitter with Contextual Book Suggestions and promised that I would follow up with a post on the type of traffic I received. Not only did the article get to be pretty popular with the Hacker News crowd, republished in Silicon Alley Insider, and make me the recipient of a handful of wonderful emails but I even got to visit tracked.com's engineering team and get schooled on machine learning techniques and A.I. (hi guys!)

Something relevant I should mention is that, just a day before, I had migrated my Posterous blog from one domain to its current place, blog.charleshooper.net. To anyone who's curious, this was a totally painless process. Set up DNS first, update your Posterous settings, and set up 302 redirects so your links don't break.

Social Link-Sharing
Despite finishing my article at 1:00 AM, I thought it was pretty well-written and I wanted my story to be heard, so I decided to get a full night's sleep, proof-read it in the morning, and submit the link to Reddit, Digg, and Hacker News. Besides the obvious benefits of proof-reading the article while fully-rested, I also recognized that most link-sharing sites weight votes based on time (or rather, some product of votes/time maybe) so submitting my article in the morning meant that it would first show up at the beginning of what I believe to be peak usage. So how'd I do?

Reddit hated it.

Digg didn't care.

And Hacker News loved it!

I also make use of Posterous' "autopost" features and have all my new submissions get posted to Twitter and Facebook. According to Posterous, there were over 77 retweets of my article (most likely from the tweet storm that the Silicon Alley Insider bot and HN bots started) and 7 "likes" on Facebook.

Traffic
According to Google Analytics, I'm not very popular. Before publishing my article, I received an average of about 30 visits a day. On the day I published my article, I observed a surge of over 4,600 visitors. From there, the numbers declined daily at a rate that looks very much like exponential decay. It took 10 days for my traffic to return to normal, but that day was a Sunday and the following Monday was almost twice as high (62 visits). For the number geeks, the set of numbers beginning with the peak is [4652, 2688, 1065, 452, 206, 138, 105]. I got close with the expression f(x) = 4652e^(-0.6x) but that isn't quite right (maybe I should treat my average visits as a constant.)

Update: I've gotten much closer with f(x) = 2658e^(-0.94x) + 30

Over the time period, my article received over 9,000 unique page views with a bounce rate of over 90% (~93% exit rate.) I remember reading an article a little while ago that stated, on single-page use cases, the bounce rate will always be close to the exit rate unless the analytics software "phones in" after some period of time to register the visit as something other than a bounce. I use Google Analytics and, unless I find out otherwise, I don't think it does this (although, it does measure "time on page" so maybe it does and Google's idea of a bounce is different than mine.)

Two winters ago I left a position as a system administrator that was paying pretty well and moved cross-country to a region with less jobs than where I moved from. Three months later, I was still unemployed, broke, and bored. I was talking to my good friend Japhy on IRC one day and he was explaining to me how the tf-idf algorithm works. For reasons involving boredom more than any other reason, I dreamed up an idea: I would write software that would take a given document and generate book suggestions based on its content.

I think that most programmers would agree with me that we put in longer hours on code when we're not working for anybody. We don't stop learning, either. To us, unemployment is a brief sprint of academia spent in our home office, the local coffee shop, or our parent's house.

My imagination dreamed up this fairly straightforward process:

1. Take a given document and calculate tf-idf scores on all terms
2. Select X number of the highest scoring terms
3. Pass these high-scoring terms to an Amazon ItemSearch query
4. Receive a list of recommended books (with URLs) from Amazon

I had already written multiple Twitter bots by this time so I decided to just use some of my existing code to poll Twitter's search API. Essentially, the "documents" I mentioned above were actually tweets containing the terms "book" or "books." Two and a half days later I had a working prototype that could generate a book recommendation from a given tweet. It was at this time that I added steps 5 and 6:

Four months later and I had generated over $7,000 in sales for Amazon with over $400 commission for myself. Obviously, the commission I was making wasn't livable but it was a nice addition to my then-depleting savings. Had I decided to scale out my operation, I could have made much more. My benchmark is at four months because that's how long I went before being suspended. My conversion rate? 0.13%! While seemingly low, this number is very high when compared to email spam. However, it's important to note that email spam is subject to various filtering technologies.

A fair amount of the time I share this story, people are more impressed with the fact that I went 4 months before getting suspended. The truth is, I had a lot of throttling built into my spam bot. The factors I think are important to point out are:

1. Twitter's Terms of Service at that time basically only outlawed "unsolicited replies," nothing that really attacked targeted spam.
2. Twitter's anti-spam stance did exist in writing (only in the help site,) but I do not think they were actively enforcing their policies.
3. My recommendations were contextual and, unless you looked at my bot's timeline and tweet count, looked legitimate (most of the time.) In other words, I was tweeting book suggestions to people who were already talking about books.
4. I recorded the usernames of everyone I sent recommendations to and would only @mention them once.
5. I built in a "chattiness" rate limiting function. This was to distribute my spam throughout a whole hour (due to Twitter's rate limiting) more than anything.

While it only lasted a short while, I had alot of fun and made a little bit of money spamming Twitter.

The second re-incarnation of this project turned into BookSuggest, a website for recommending books based on a person's Twitter feed. I haven't put alot of effort into promoting it, but my conversion rate is much lower now that I'm not pushing the links in anyone's face.

Try it out and comment here - what did BookSuggest tell YOU to read?

As I stand before you sipping my celebratory Wilco Tango Foxtrot, I would like to present to you the new home for my blog. The reason for this migration is that I am downsizing the amount of under-utilized domains I renew each year. Please update your bookmarks accordingly :-)

Previously, we discussed the various regulations and regulatory bodies that govern financial reporting. We will now turn to the Generally Accepted Accounting Principles (GAAP) to explain the basic principles used in accounting. In particular, we will discuss the cost, revenue recognition, matching, and full disclosure principles.

While it may sound redundant, the cost principle means that “accounting information is based on an actual cost” (Wild, Shaw, & Chiappetta, 2009, p. 9). In other words, everything is treated as having the value of what was paid for it. So what happens when businesses make a trade or don't purchase with cash (businesses have been known to buy each other with a mix of cash, stocks, and bonds)? “If something besides cash is exchanged … cost is measured as the cash value of what is given up or received” (Wild, Shaw, & Chiappetta, 2009, p. 10). That caveat here is that if you buy something and get a good deal, such as buying a $7000 asset for $5000, the item will be recognized in your accounting system as having a value of $5000. This is to ensure that the accounting information remains objective (Wild, Shaw, & Chiappetta, 2009, p. 10). Next, we will look at the revenue recognition principle.

The revenue recognition principle determines how and when a company will recognize (record) revenue (Wild, Shaw, & Chiappetta, 2009, p. 10). The primary concept of the revenue recognition principle is that “Revenue is recognized when earned” (Wild, Shaw, & Chiappetta, 2009, p. 10). This doesn't necessarily mean when the customer or client pays for their good or service, but when the good or service is actually sold (such as on credit). For example, if I configured someone's network (a service) at an hourly rate, I would be required to recognize and record this earned revenue as soon as the work is done; this is usually done crediting accounts receivable (most accounting software does this automatically when generating an invoice.) This principle is intended to keep companies from recognizing revenue too early to look more profitable while also ensuring that they don't recognize revenue too late to look less profitable than they really are (Wild, Shaw, & Chiappetta, 2009, p. 10). Now let's look to the matching principle.

The matching principle dictates that a company must report its expenses in the period that they generated the revenue reported (Wild, Shaw, & Chiappetta, 2009, p. 10). Let's say, for example, that a company buys 10 pounds of raw material, uses it to make 10 widgets, and then sells 5 of those widgets. Under the matching principle, the company would report the expenses (or, in this case, Cost of Goods Sold) incurred to make the 5 widgets it sold. The remaining 5 (that are now sitting in inventory), would not have their expenses/COGS reported until they too were sold. Finally, we turn to the full disclosure principle.

The full disclosure principle is probably the most basic yet most important principle. The full disclosure principle states that a company must “report the details behind financial statements that would impact users' decisions” (Wild, Shaw, & Chiappetta, 2009, p. 10). Oftentimes, these details are reported in footnotes of a company's financial statements or annual reports (Wild, Shaw, & Chiappetta, 2009, p. 10). An example of this from current events is Dell's recent trouble with the SEC. Dell's recent trouble was partially the result of receiving money from CPU manufacturer Intel and disguising that money as sales (why they hid it is another topic entirely). Users of Dell's financial reports were led to believe that this extra money was the result of sales. When Intel stopped paying Dell this “incentive money,” Dell then took extra steps to falsify their financial statements to hide the fact that their revenue decreased. Save for the anti-trust violation with Intel, if Dell had just fully disclosed the revenue it was receiving from Intel they may have never felt the pressure to hide the fact that the payments stopped.

In conclusion, the Generally Accepted Accounting Principles (GAAP) are made up of four basic principles. In particular, we discussed the cost, revenue recognition, matching, and full disclosure principles.

Chiappetta, B., Shaw, K., Wild, J. (2009). Principles of Financial Accounting (19^th ed.). McGraw-Hill/Irwin.

After Enron, Global Crossing, and Xerox, we witnessed the creation of new regulation (the Sarbanes-Oxley Act of 2002), the establishment of a new oversight/regulatory organization, and increased pressure from the government on big players in the market. As we find ourselves in the midst of the Financial Crisis of 2007 (and onward,) we are seeing this same pattern – new legislation, a new regulatory organization, and increased pressure from the government on big players in the market.

In news today, we see the Restoring American Financial Stability Act of 2010 and the Wall Street Reform and Consumer Protection Act of 2009 on the desk of the President. This 2,300-page bill will “ban high-risk trading inside the banks and put in end to conflicts of interest” and “ban steering payments, liar loans, and prepayment penalties and give Americans … transparency” (Nichols, 2010, para. 4-5). But that isn't all. These new acts would also create yet another oversight/regulatory organization called the Consumer Financial Protection Bureau to regulate mortgages, credit cards, and other financial consumer products (Hall, 2010, fig. 1). Not only does this act establish a new bureau, it also creates a “council of regulators” led by the Treasury to monitor “threats to the financial situation” (Hall, 2010, fig. 1).

And as if you couldn't already guess, today in the news we see increased pressure from the government on big players in the market including Goldman Sachs, Dell, and AIG. Goldman Sachs agreed to settle by paying a $550 million dollar fine – the largest fine ever charged by the SEC after being accused of securities fraud (Craig, Scannel, 2010, para. 3). Dell admitted to accounting fraud back in 2007 and agreed on a settlement with the SEC today – The company must pay a $100 fine and restate its earnings for the 2003 to 2006 accounting periods (Ogg, 2010, para. 2-3). What Dell was caught doing was violating GAAP, the Global Accepted Accounting Principles, by “fudging” the timing of recognized expenses and income to meet analyst's quarterly earnings forecasts (Ogg, 2010, para. 1,4). And if that isn't enough, the AIG settled with the Ohio State Attorney General with a $725 million fine (AFP, 2010, para. 1). The AIG was also accused of accounting fraud to achieve stock price manipulation (AFP, 2010, para. 2).

As we can clearly see, after each financial crisis the response is likely to be the same.

AFP. (2010). AIG to Pay 725 Million Dollars to Settle US Fraud Lawsuit. Retried from http://www.google.com/hostednews/afp/article/ALeqM5h0MtX-uuAB1z2f3dcmONOLRCIWaw

Craig, Susanne, Kara Scannel. The Wall Street Journal. (2010). SEC Split Over Goldman Deal. Retrieved from http://online.wsj.com/article/SB10001424052748704229004575371601322076426.html?mod=WSJ_hpp_LEFTTopStories

Hall, Kevin. McClatchy Newpapers. (2010). What's this big finance-regulation overhaul really do? Retrived from http://www.mcclatchydc.com/2010/07/15/97609/whats-this-big-finance-regulation.html

Ogg, Erica. CNET. (2010). Dell to Restate Earnings Due to Accounting Fraud. Retrieved from http://news.cnet.com/8301-31021_3-20007390-260.html

Ogg, Erica. CNET. (2010). Goals Led Dell to Cook the Books. Retrieved from http://news.cnet.com/Goals-led-Dell-to-cook-the-books/2100-1014_3-6203071.html?tag=mncol;txt

Nichols, John. The Nation. (2010). Financial Reform Passes, But What Does That Mean? Retrieved from http://www.thenation.com/blog/37541/financial-reform-passes-what-does-mean

Editor's note: I wrote this one kind of late. It's not my best, but I did get a pretty good grade on it.

Enron was an energy trading company. Rather than generate energy themselves, Enron made its money by buying and selling energy contracts. At one point, Enron was considered one of the most innovative companies on the market and reported sales of over $100 billion one year. Soon after, the company then reported losses of $618 million loss in the third quarter of 2001 (Nielsen, 2002, para. 1). A little later, the firm filed for Chapter 11 bankruptcy. “The company's Chapter 11 filing [left] banks, pension plans and other lenders with at least $5 billion at risk. More than 4,000 Enron employees lost their jobs and 401(k) savings. The collapse reverberated the stock market, which dropped some $200 billion in value since Enron's Dec. 2 filing, amid fears that other Enrons are lurking out there” (Jaffe, 2002, para. 3).

So how does a company go from reporting over $100 billion dollars in sales when they are really on the verge of bankruptcy? The answer lies in the deceptive accounting practices and outright fraud that Enron's executives and auditors took part in. One factor is the massive amount of debt that Enron was hiding from its balance sheet. How does one go about hiding debt? “At the heart of Enron's demise was the creation of partnerships with shell companies, many with names like Chewco and JEDI, inspired by Star Wars characters. These shell companies, run by Enron executives who profited richly from them, allowed Enron to keep hundreds of millions of dollars in debt off its books” (Nielsen, 2002, para. 4). At the same time, Enron's board members, regulators, analysts, auditors, and even politicians looked the other way while Enron committed its fraud (Kadlec, 2001, para. 4). Enron's auditing firm even went as far as to order their employees to shred all of the documents they used to do Enron's audits! In the wake of Enron's collapse, the general public demanded that the politicians step up, create, and enforce corporate law. This allowed for the creation of the Sarbanes-Oxley Act of 2002 which is still in effect today!

Jaffe, Stephen. TIME. (2002). How Fastow Helped Enron fall. Retrieved from http://www.time.com/time/business/article/0,8599,201871,00.html

Kadlec, Daniel. TIME. (2001). Power Failure. Retrieved from http://www.time.com/time/magazine/article/0,9171,1101011210-186639,00.html

Nielsen, James. TIME. (2002). Enron: Who's Accountable? Retrieved from http://www.time.com/time/business/article/0,8599,193520,00.html

It occurred to me that I haven't spent much time writing about one of my most important projects: NLON Labs. NLON Labs, named for its hometown New London, is a members-run community workshop, or Makerspace. We initially started out back in March as an ongoing email thread between two people, which turned into three people, then four, and then turned into a mailing list of its own. By April we had already Incorporated as a Connecticut non-stock corporation and legally organized. By May we were moving into our new space, much thanks to our new landlord, the local Garde Arts Center.

The Sarbanes-Oxley Act of 2002, also known as the SOX Act, was created in response to the series of misleading and outright fraudulent activity of big business in the 1990s (Lasher, 2008, p. 187). Essentially, multiple publicly-traded businesses jacked up their stock prices by “publishing false or deceptive financial statements” (Lasher, 2008, p. 187). The most notable company to crash was Enron, followed by Global Crossing (parent of MCI,) and Xerox; later, almost one thousand publicly traded companies restated their financial statements (Lasher, 2008, p. 187). This resulted in almost $6 trillion of stock market value disappearing (Lasher, 2008, p. 187)! In response to these events, Congress drafted and passed the Sarbanes-Oxley Act (SOX) of 2002.

Next, we look to the provisions of the Sarbanes-Oxley Act of 2002 to see how they impact the way firms prepare their financial statements. The first and most major way the SOX Act impacted financial reporting was that it ended self-regulation of the public accounting industry (Lasher, 2008, p. 190). The SOX Act achieved this by establishing the Public Company Accounting Oversight Board (PCAOB,) an independent, non-profit organization (Lasher, 2008, p. 190). The PCOAB is given authority and empowered by the Securities Exchange Commission (SEC) to regulate and enforce these regulations of the accounting industry (Lasher, 2008, p. 190). Under the regulations of the SOX Act and the PCOAB, it is now required for all accounting firms to be registered and illegal for an unregistered firm to issue audit reports for publicly-traded companies (Lasher, 2008, p. 190). The PCOAB is empowered and directed to perform investigations of questionable accounting practices, hold disciplinary hearings, and impose sanctions upon firms and individuals who auditors who are caught letting wrongdoings “slide” (Lasher, 2008, p. 190). Another way the SOX Act effectively restores the integrity of financial statements is by removing a very large conflict of interest that existed in the 1990s. This conflict of interest existed for accounting firms that also provided consulting services to their clients; under the SOX Act, these types of relationships became illegal and firms that had audit clients could no longer offer these same clients additional consulting services (Lasher, 2008, p. 190). Another major conflict of interest that was removed was the relationships between auditors and the audited firm's CEO and CFO. Instead of reporting to the firm's executives, auditors now reported to an “audit committee” of the firm's Board of Directors; At least one member of which is legally required to be a financial expert (Lasher, 2008, p. 190). Finally, partners of an auditing firm can only supervise the same client for five years at a time (Lasher, 2008, p. 191). The final main problem that the SOX Act addresses is the potential conflict of interest between a firm and its executives, also known as the agency problem (Lasher, 2008, p. 192). The SOX Act achieved this by requiring CEOs and CFOs to certify that they reviewed their firm's financial statements, that they are true to the best of their knowledge, to also certify that they are personally responsible for their firm's internal financial controls, and by requiring that company executives repay bonuses and capital gains from stock sales if they follow within 12 months of the issue of financial statements and the gains are made as a “result of misconduct” (Lasher, 2008, p. 191).

Lasher, William R. (2008). Practical Financial Management (5^th ed.). Thomson South-Western.

I had trouble getting the subprocess module onto BackTrack 4. apt-get install python-subprocess wouldn't do it, nor would easy_install subprocess. As it turns out, the name of the backported subprocess module is subprocess32, so easy_install subprocess32 fixed me up real good.