tag:blogger.com,1999:blog-21627703744030668262024-03-12T17:38:03.519-07:00OasicUnknownnoreply@blogger.comBlogger19125tag:blogger.com,1999:blog-2162770374403066826.post-77465392271963618612013-11-18T08:41:00.004-08:002013-11-18T08:41:33.885-08:00Search API Notes<span style="font-size: large;"><b>Scenario:</b> <i>You've got a great idea that requires indexing and/or search capabilities well beyond your budget. Where do you go from here?</i></span><br />
<br />
Thankfully, you have a few options to choose from when deciding how to power your new app. Sadly, you have ONLY a few options to choose from. Indexing and searching the Internet is a monstrous task, which is why this industry is a natural fit for the oligopoly we see today. There are three players in this market that all offer Search APIs, but as of this writing, their products differ considerably.<br />
<br />
<b>Yahoo BOSS</b> - <a href="http://developer.yahoo.com/boss/search/">http://developer.yahoo.com/boss/search/</a><br />
If you are looking for something inexpensive, then this is it. They offer a 'limitedweb' search that is slightly smaller and not as fresh, but it's only $0.40/1000 queries, which is half the price of their 'web' offering. Other than the cost savings, this service stinks. Do not use this unless your application allows for a large margin of error and cost is the most important requirement. I've found 3 types of common problems:<br />
- False positives: returning results that do not contain the query. It doesn't matter whether you are using an exact phrase search, boolean operators, etc. Regardless, you will get false positives from time to time.<br />
- False negatives: matching results that are in Yahoo's index fail to be returned sometimes<br />
- Sporadic errors: the errors mentioned above, as well as other outages, occur frequently and randomly. While developing with this API it was very frustrating because it does not return consistent results. The same query will return no results one minute, then many results a minute later. Frustrating.<br />
Bottom line: DO NOT USE ON IMPORTANT WORK<br />
<br />
<b>Google</b> - <a href="https://developers.google.com/custom-search/">https://developers.google.com/custom-search/</a><b><br /></b><br />
On the other end of the spectrum is the dominant search giant. Their API is high-quality and VERY expensive ($5/1000 queries). Notice that that is more than 10X the cost of Yahoo's limitedweb queries. Nevertheless, the Google results are consistent and of the quality you would expect.<br />
Disadvantages: Besides price, Google's API results often do not match their public search results. If you have a high volume app, the rate limits may be a deal-breaker for you (it was for us).<br />
<br />
<b>Microsoft Bing</b> - <a href="http://datamarket.azure.com/dataset/8818F55E-2FE5-4CE3-A617-0B8BA8419F65">http://datamarket.azure.com/dataset/8818F55E-2FE5-4CE3-A617-0B8BA8419F65</a><br />
I'm rarely a fan of anything Microsoft produces, but they are the winner in my evaluation of web search APIs. They have just the right mix of consistency, price, and performance, without the restrictions of Google. They offer unlimited searches at a price that is roughly $1.25/1000 queries. This is 1/4 of Google, but still 3X more than Yahoo's limitedweb. For mission critical apps that can't afford the problems of BOSS, Bing is probably the best choice. Be sure to use the "Web Only" API if you are only using their web search, as it is cheaper than their composite search offering.<br />
<br />
<br />
<br />Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-2162770374403066826.post-59614472180019787082013-08-29T11:27:00.001-07:002014-05-29T08:49:34.865-07:00Resources for the Internet of ThingsWhether you refer to it as The Internet of Things, the Sensor Revolution, or the Programmable World, here are some potentially useful resources:<br />
<ul>
<li><a href="http://www.openremote.org/display/docs/Get+Started" target="_blank">OpenRemote</a> </li>
<li>SmartThings - <a href="http://build.smartthings.com/" target="_blank">Developers</a> and <a href="https://shop.smartthings.com/#/products" target="_blank">Store</a></li>
<li><a href="http://www.makerswarm.com/" target="_blank">MakerSwarm</a> </li>
<li><a href="http://razberry.z-wave.me/" target="_blank">RaZberry</a> - Z-wave for Raspberry Pi</li>
<li><a href="http://www.nimbits.com/" target="_blank">Nimbits</a> </li>
<li><a href="http://www.iobridge.com/" target="_blank">ioBridge</a> - complete stack </li>
<li><a href="http://ninjablocks.com/" target="_blank">NinjaBlocks</a> - sensors, devices, and programmable hub </li>
<li><a href="http://www.limitlessled.com/shop/limitless-led-starter-kit/" target="_blank">Limitless LED</a> - programmable lights</li>
<li><a href="http://www.belkin.com/us/Products/home-automation/c/wemo-home-automation/" target="_blank">WEMO</a> - Belkin home automation devices</li>
<li><a href="http://www.alyt.com/" target="_blank">Alyt</a> - Android-based home security and automation hub & devices </li>
</ul>
<b>Microcontrollers & Systems on a Chip </b><br />
<ul>
<li><a href="http://www.tinkerforge.com/en" target="_blank">TinkerForge</a> - Stackable, programmable boards. Appears to be simpler than Arduino in many ways.</li>
<li>Arduino - Everyone's favorite open-source microcontroller</li>
<li>BeagleBone Black</li>
<li>Raspberry Pi</li>
<li><a href="http://logiblock.com/" target="_blank">Galago</a> - </li>
<li><a href="http://arduino.cc/en/ArduinoCertified/IntelGalileo" target="_blank">Intel Galileo</a> - open-source board from Intel</li>
</ul>
<br />
<br />
I will try to expand this list over time. If you have more, please add them in the comments. Thanks!Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-2162770374403066826.post-57076899743858052102013-01-03T12:30:00.000-08:002013-10-23T11:41:02.063-07:00Will a Robot Take My Job?<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipLkx-lVzzLw5yP3hq3aPDycCGxaJW_3IymwsHG5RIwRhmV45EEtsx2-PHjMF-pXXc0O5ndFdoPd66OT23m5faIcl_Wy2RX-pe_jFAnk75bRsXgp2bjt0k0OJL_5N0yl34sdOwp8vwPTo/s1600/jhen.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipLkx-lVzzLw5yP3hq3aPDycCGxaJW_3IymwsHG5RIwRhmV45EEtsx2-PHjMF-pXXc0O5ndFdoPd66OT23m5faIcl_Wy2RX-pe_jFAnk75bRsXgp2bjt0k0OJL_5N0yl34sdOwp8vwPTo/s1600/jhen.jpg" /></a>With many jobs lost and the economy teetering on recovery, this question couldn't be more relevant. The man vs machine debate has a long history and conjures up images of John Henry racing to his death against a steam hammer. The reality is that machines already do the work of people and will continue to usurp greater and greater responsibilities. If your job is not one of the ones lost, then it is quite easy to see that this expansion should be welcome as it allows for greater societal productivity. After all, do you miss doing dishes by hand? Wish you could wash your laundry in a tub and hang it out to dry? Want to pay greater prices for hand made products? The key is understand <i><b>which</b></i> jobs are likely to be filled by machines. The following 5 questions should help.<br />
<br />
<b> </b><br />
<br />
<b>Is your work highly repetitive?</b>
<b> </b><br />
Specialized, repetitive tasks are easier to automate. It is the human ability to generalize that makes our human "wetware" different than robotic hardware. Maybe you execute the same movements as part of a manual labor position, or maybe you spend the day downloading the same data and plugging it into the same spreadsheet. Whatever the case may be, if you find yourself doing the same tasks over and over, then your job is more likely to be taken by a machine.<br />
<br />
<br />
<b>Are there many people in your company in the same role as you?</b><br />
If so, you are a great target for automation. Developing algorithms and/or hardware to replace one person is often not cost effective, but it may be a no-brainer for a company to invest in machines that replace 100 people.<br />
<br />
<br />
<b>What collar do you wear? White or blue?</b><br />
Previous generations of machines have found great success replacing manual labor in factories, but the next generation of machines will be more likely to replace office workers -- white collar folks. The reason for this is that massive data and computing power have reached a critical mass allowing difficult "thinking" tasks to be conquered by machines. We now have machines that can dominate Jeopardy, diagnose cancer from a breast scan, and grade essays. This trend will only increase as more data and computing power become available in the years ahead. <br />
<br />
On the other hand, most jobs that involve service work or manual labor will be difficult for robots to replace. The reality is that complex analytical thinking will be easier to duplicate than our nerves and muscles. We take for granted how easily we handle sophisticated movements, but it will be many years before robots are equipped with the sensors and actuators comparable to a human's. Until then, don't expect the moving company to send in robots to pack up your house.<br />
<br />
<br />
<b>Do you interact with people?</b><br />
Many jobs require a uniquely human touch, something that machines do not offer -- something that machines may never offer. If meaningful interaction with people is an important part of your job, then you are unlikely to be replaced by a machine any time soon. Don't expect to be seeing a salesperson or a robotic shrink in the next century.<br />
<br />
<br />
<br />
<br />
The bottom line is that the use of machines will continue to allow society to be more productive, and productivity often means one person doing the work of many. We should embrace this new potential, but also be cognizant of the skill sets most needed in the 21st century.<br />
<br />
<br />Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-2162770374403066826.post-41452254781726628772012-09-12T13:12:00.000-07:002012-09-12T13:14:05.056-07:00Quick Solutions: Rails 3, send_data, and garbled PDF outputThis is just a quick blog entry to help anyone experiencing garbled PDF output (in Safari) when using Rails 3's send_data to output dynamically-generated PDF files. <br />
<br />
If you are following the documentation, then you are probably outputting something like this:<br />
<br />
<pre class="brush:ruby">
send_data data, :filename => "myfile.pdf",
:type => 'application/pdf'
</pre>
<br />
I don't know if this is specific to Rails 3, but the issue is that the Content-Type header is not being set to 'application/pdf', so setting it explicitly in the response should fix this issue:<br />
<br />
<br />
<pre class="brush:ruby">
response.headers["Content-Type"]='application/pdf'
send_data data, :filename => "myfile.pdf",
:type => 'application/pdf'
</pre>Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-2162770374403066826.post-36921944425903371102012-06-26T20:25:00.001-07:002012-06-28T06:52:40.184-07:00Machine Learning: Naive Bayes Classification with RubyMaybe you've wondered, "<i>Where are all the Ruby libraries for Machine Learning and NLP?</i>". Despite Ruby's growing user base and ability to quickly manipulate data and text, there seems to be a dearth of tools for NLP and Machine Learning. One statistical tool that finds itself in the intersection of NLP and Machine Learning is Naive Bayes. <span style="font-size: x-small;">[For those already familiar with Naive Bayes, you may wish to skip ahead to the Quickstart section below (although I promise not to be long-winded in my intro).] <span style="font-size: small;"> </span></span><br />
<br />
<span style="font-size: x-small;"><span style="font-size: small;">This peculiarly-named approach to <a href="http://en.wikipedia.org/wiki/Statistical_classification" target="_blank">classification tasks</a> is based on the well-known <a href="http://en.wikipedia.org/wiki/Bayes%27_rule" target="_blank">Bayes' Rule</a> and is used to calculate the probability an instance belongs in a particular class (or category) based on the components of the instance. It is called "<b>Naive</b>" Bayes because of the manner in which it calculates the probabilities. It treats each component as if it is independent from other components, even though this is usually not the case. Suprisingly, Naive Bayes does quite well and is optimal in the case in which the components of an instance actually are independent. </span></span><br />
<br />
<span style="font-size: x-small;"><span style="font-size: small;">Enough generalities...Naive Bayes is used widely for text classification problems and the "components" that I referenced above are actually tokens -- typically words. Even if you have no desire to understand the probabilistic engine beneath the hood, Naive Bayes is easy to use, high performance, and accurate relative to other classifiers. It requires a 2 step process:</span></span><br />
<br />
<span style="font-size: x-small;"><span style="font-size: small;">1) Train the classifier by providing it with sets of tokens (e.g., words) accompanied by a class (e.g., 'SPAM')</span></span><br />
<span style="font-size: x-small;"><span style="font-size: small;">2) Run the trained classifier on un-classified (e.g., unlabelled) tokens and it will predict a class</span></span><br />
<br />
<span style="font-size: x-small;"><b><span style="font-size: small;">Quickstart </span></b></span><br />
<br />
gem install nbayes<br />
<br />
After that, it's time to begin training the classifier:<br />
<pre class="brush:ruby"># create new classifier instance
nbayes = NBayes::Base.new
# train it - notice split method used to tokenize text (more on that below)
nbayes.train( "You need to buy some Viagra".split(/\s+/), 'SPAM' )
nbayes.train( "This is not spam, just a letter to Bob.".split(/\s+/), 'HAM' )
nbayes.train( "Hey Oasic, Do you offer consulting?".split(/\s+/), 'HAM' )
nbayes.train( "You should buy this stock".split(/\s+/), 'SPAM' )
</pre>
<br />
<br />
Finally, let's use it to classify a document:<br />
<br />
<pre class="brush:ruby"># tokenize message
tokens = "Now is the time to buy Viagra cheaply and discreetly".split(/\s+/)
result = @nbayes.classify(tokens)
# print likely class (SPAM or HAM)
p result.max_class
# print probability of message being SPAM
p result['SPAM']
# print probability of message being HAM
p result['HAM']
</pre>
<br />
<br />
But that's not all! I'm claiming that this is a full-featured Naive Bayes implementation, so I better back that up with information about all the goodies. Here we go:<br />
<br />
<b>Features
</b><br />
<ul>
<li>Works with all types of tokens, not just text. Of course, because of this, we leave tokenization up to you.</li>
<li>Disk based persistence </li>
<li>
Allows prior distribution on classes to be assumed uniform (optional)<br />
</li>
<li>
Outputs probabilities, instead of just class w/max probability<br />
</li>
<li>
Customizable constant value for Laplacian smoothing<br />
</li>
<li>
Optional and customizable purging of low-frequency tokens (for performance)<br />
</li>
<li>
Optional binarized mode to reduce the impact of repeated words<br />
</li>
<li>
Uses log probabilities to avoid underflow<br />
</li>
</ul>
<br />
I hope to post examples of these features in action in a future post. Until then, view nbayes_spec.rb for usage.<br />
<br />
<br />
<br />Unknownnoreply@blogger.com7tag:blogger.com,1999:blog-2162770374403066826.post-5039831581417291002012-04-16T11:06:00.002-07:002012-04-16T11:11:44.854-07:00Solutions to invalid byte sequence in UTF-8 (ArgumentError)Let's keep this short and simple. You're reading this because you too are tired of all the new character set issues in Ruby 1.9. This page lists some possible solutions and links to other pages with helpful info on solving "invalid byte sequence in UTF-8 (ArgumentError)".
<b> </b><br />
<br />
<b>The Cause</b><br />
This problem occurs when your application is expecting one character encoding and gets a different one.
<b> </b><br />
<br />
<br />
<b>Possible Solutions:</b><br />
<br />
1) Explicitly specify the encodings that you expect in the top-level Ruby Encoding class. Put this in a file that will be loaded (e.g., boot.rb for Rails):
<br />
<pre><blockquote>
Encoding.default_external = 'UTF-8'
Encoding.default_internal = 'UTF-8'
</blockquote>
</pre>
<br />
2) Convert the character encoding on a lower level within your application
<br />
<pre><blockquote>
ascii_str.encode("UTF-8")</blockquote>
</pre>
More info: http://blog.grayproductions.net/articles/ruby_19s_string<br />
<br />
<br />
3) Finally, if the text you are working with has a mix of character encodings (occasionally this happens when data is aggregated improperly), then you may need to decide what is the most common character set and convert from that to the new char set (e.g., UTF-8), while ignoring any other unrecognizable characters.<br />
<br />
You can do this using iconv from the command line or from within Ruby:<br />
within Ruby:
<br />
<pre><blockquote>
ic = Iconv.new('UTF-8//IGNORE', 'UTF-8')
valid_string = ic.iconv(untrusted_string)
</blockquote>
</pre>
(source: http://smyck.net/2011/05/13/files-with-mixed-and-invalid-encodings-in-ruby/)<br />
<br />
command line:
<br />
<blockquote>
<pre>iconv -f WINDOWS-1252 -t "UTF-8//IGNORE" some_text.txt > some_text.utf8.txt
</pre>
</blockquote>
<br />
4) Catch the error and handle it more gracefully:
<br />
<pre><blockquote>
begin
# your code
rescue ArgumentError => e
print "error: #{e}"
end
</blockquote>
</pre>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-2162770374403066826.post-72906330508882588532012-01-19T13:02:00.000-08:002012-01-19T13:02:43.582-08:00Taking a Random Walk with Processing 1.5<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4LQNCQ8JYuDM3_SGeOXhEOOIjOp4f71_S6U2K_Z7LJBs3Woez2kozj97-O1pTw_gUsDROh7OZLVT2vkBnkiWxvOVhwK3U5_SOcpOlcyi24-QHU8O_vSgWwVY2j8No6uzCAaNwlhlQw7k/s1600/random_walk.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4LQNCQ8JYuDM3_SGeOXhEOOIjOp4f71_S6U2K_Z7LJBs3Woez2kozj97-O1pTw_gUsDROh7OZLVT2vkBnkiWxvOVhwK3U5_SOcpOlcyi24-QHU8O_vSgWwVY2j8No6uzCAaNwlhlQw7k/s320/random_walk.png" width="320" /></a></div>
If you've ever dabbled with simulations you have probably come across <a href="http://www.processing.org/">Processing</a>, the open source environment for animation, interaction, and much more. The Java-ish scripting language allows for quick prototyping and the <a href="http://processing.org/learning/topics/">library of examples</a> allow even beginners (me) to get moving quickly.<br />
<br />
To the right is an image I created from a very simple simulation of a "random walk" along the y-axis as time moved along the x-axis. Many have noted the similarity between this process and the peaks and valleys of markets, mountain ranges, and other stochastic processes. If you haven't already, give it a try.<br />
<br />
Source for Random Walk:<br />
<pre>int x, y, r, middle, randOffset, previousX, previousY;
void setup() {
size(900, 900);
stroke(0);
background(192, 64, 0);
x = 0;
r = 2;
y = 200;
}
void draw() {
previousX = x;
previousY = y;
x++;
// y does random up or down of 10 units
randOffset = 10 - (int)random(21);
y = y + randOffset;
//line(150, 25, mouseX, mouseY);
stroke(0);
line(previousX, previousY, x, y);
if(x < 800) save("random_walk.tif");
}
</pre>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-2162770374403066826.post-45128702464435297602011-08-17T13:38:00.000-07:002011-08-17T13:38:02.695-07:00Rails 3, Ruby 1.9.2, and Character Encoding Nightmares<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiq-SyCREBPP-bNYKmiHCCa0Eyvu-sTETqwBf1s5g3iLmAZTjvV8qXVDZmwA0-SHqBS_xAqNMvATdzcDNXj8dV6q_W0nndlVxiXaY9n2RjFnG-MLvyxNw0VlmmpnXBzzfF2u6dcG-VrOp4/s1600/resized_tower_of_babel_19_jun_091.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiq-SyCREBPP-bNYKmiHCCa0Eyvu-sTETqwBf1s5g3iLmAZTjvV8qXVDZmwA0-SHqBS_xAqNMvATdzcDNXj8dV6q_W0nndlVxiXaY9n2RjFnG-MLvyxNw0VlmmpnXBzzfF2u6dcG-VrOp4/s200/resized_tower_of_babel_19_jun_091.jpg" width="146" /></a></div>Ruby (and Rails) can't take all the blame for this mess. No, it's been present as long as humanity has grasped the social innovation of language. Even if your goals are not quite so high as the Tower of Babel (perhaps just writing a cool web app), it's understandable to feel some sort of divine opposition when dealing with frustrating character set issues. Now that Ruby 1.9 and Rails 3 have become wise to character encodings, the reality of dealing with this messy subject is playing out in web applications and wreaking havoc on many. Having dealt with these problems in many web applications from Java to Ruby over the years, it's easy for me to think that immunity is certain, even deserved, but think again.<br />
<br />
<b style="color: #0c343d;">The Problem Described </b><br />
Just recently, a very popular website that I consult on was occasionally throwing the following error that is not uncommon after upgrading to Rails 3 and Ruby 1.9.x:<br />
<br />
<blockquote>ActionView::Template::Error: incompatible character encodings: ASCII-8BIT and UTF-8</blockquote><br />
<br />
<br />
I've received this type of error (incompatible character encodings) before, but not in a Rails view. Nevertheless, I began by inspecting the usual suspects.<br />
<br />
MySQL: <b><span style="font-size: x-small;"><i>SHOW VARIABLES LIKE '%char%';</i></span></b><br />
Ok, all clear there...<br />
<br />
application.rb includes config.encoding="UTF-8" -- <span style="background-color: lime;">YES</span><br />
using 'mysql2' gem -- YES<br />
setting # encoding: utf-8 at the top of any ruby files with UTF-8 string literals -- <span style="background-color: lime;">YES</span><br />
<br />
running 'locale' on my linux system...UTF-8... -- <span style="background-color: lime;">YES</span><br />
firing up the Rails console and printing out Encoding.default_internal and Encoding.default_external... -- <span style="background-color: lime;">YES</span><br />
<br />
If you reach this point and you are still receiving the problem, then it should be clear that this is probably not going to be solved easily. Fortunately, the stack trace gave me the line in question from my erb file and Airbrake (formerly Hoptoad), provided the specific input that was causing the problem. Using this, I was able to duplicate the error with a Rails functional test and begin looking for solutions.<br />
<br />
A string was being output in the view such that changing it to<br />
<%= mystring.force_encoding("ASCII-8BIT") %><br />
solved the problem, but not really. This caused the error to go away, but only by asking that Rails treat a UTF-8 string as an ASCII string, which could lead to more serious problems. And why would setting a string to ASCII fix the problem in a UTF-8 Rails app? Shouldn't that cause the problem?<br />
<br />
After further tinkering, I'll describe my current understanding of what happened; and if you're reading along, hopefully this will be helpful in allowing you to solve a similar problem that you might be having. As I will get to later, a string that was output higher up in the erb file had caused Erb to switch its encoding from UTF-8 to ASCII-8BIT. Perhaps this functionality was added to allow Erb to gracefully handle any character encoding. The problem in this case arose when 2 strings with different character encodings were being output in the same erb file. So, it was simply a matter of tracking down which other string caused the problem.<br />
<br />
As it turns out, it was a string populated from an HTTP request using Net::HTTP, which sets the encoding to "ASCII-8BIT", even when the actual encoding is "UTF-8". Since the string was a UTF-8 string mis-labelled as ASCII, the solution was to call "http_string.force_encoding("UTF-8")", which correctly labelled the string. After this change was made, all tests passed. I hope you have the same luck with your related problem. :-)<br />
<br />
<br />
<br />
Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-2162770374403066826.post-86155271158017729562011-08-08T12:51:00.000-07:002011-08-08T12:51:52.258-07:00FeedTools + Ruby 1.9.2 + Rails 3During an upgrade of a RoR app, the following was encountered:<br />
<br />
<br />
<blockquote><span style="font-size: x-small;">ruby -c /usr/local/rvm/gems/ruby-1.9.2-p0/gems/feedtools-0.2.29/lib/feed_tools/helpers/uri_helper.rb<br />
/usr/local/rvm/gems/ruby-1.9.2-p0/gems/feedtools-0.2.29/lib/feed_tools/helpers/uri_helper.rb:43: invalid multibyte char (US-ASCII)<br />
/usr/local/rvm/gems/ruby-1.9.2-p0/gems/feedtools-0.2.29/lib/feed_tools/helpers/uri_helper.rb:43: invalid multibyte char (US-ASCII)<br />
/usr/local/rvm/gems/ruby-1.9.2-p0/gems/feedtools-0.2.29/lib/feed_tools/helpers/uri_helper.rb:43: syntax error, unexpected $end, expecting ')'<br />
if IDN::Idna.toASCII('http://www.詹姆斯.com/') ==<br />
^</span></blockquote>To fix, simply add the following to the first line of uri_helper.rb in the feed_tools gem directory:<br />
<br />
# encoding: utf-8<br />
<br />
This allows the Ruby interpreter to correctly interpret the file as a UTF-8 file.<br />
<br />
<br />
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-2162770374403066826.post-71369591970769998532011-04-26T07:14:00.000-07:002011-04-26T07:14:22.417-07:00Using Mysqldump Without Downtime<div style="text-align: right;"><a href="http://www.amazon.com/Learning-MySQL-Seyed-Saied-Tahaghoghi/dp/0596008643?ie=UTF8&tag=bulldogbreeds-20&link_code=bil&camp=213689&creative=392969" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;" target="_blank"><img alt="Learning MySQL" src="http://ws.amazon.com/widgets/q?MarketPlace=US&ServiceVersion=20070822&ID=AsinImage&WS=1&Format=_SL160_&ASIN=0596008643&tag=bulldogbreeds-20" /></a></div><img alt="" border="0" height="1" src="http://www.assoc-amazon.com/e/ir?t=bulldogbreeds-20&l=bil&camp=213689&creative=392969&o=1&a=0596008643" style="border: medium none ! important; margin: 0px ! important; padding: 0px ! important;" width="1" />Mysqldump is a great utility for creating easy to use, easy to restore mysql database backups, but it can cause downtime if certain precautions are not taken. If you manage MySQL across multiple Linux flavors, various default configurations can cause mysqldump to act in unpredictable ways. Let's look at a couple common problem areas related to mysqldump.<br />
<br />
<b>Using the Right Storage Engine</b><br />
If you're not certain which MySQL storage engine to use, then consider using InnoDB (instead of MyISAM) if you are not already doing so. Using mysqldump to backup a large MyISAM table can cause the entire table to lock until the backup is complete. Even though the long-running read done by mysqldump does not block other reads of the same table, what happens is that an update query issued will cause all subsequent reads to be queued and therefore blocked. So, switching to InnoDB will solve this type of problem due to its usage of row-level locking.<br />
<br />
<b>Using --single-transaction</b><br />
On most of the MySQL installations that I've managed, simply using InnoDB will allow for backups to be created w/o tables locking and queries/updates blocking. However, a recent Ubuntu installation continued to block our Rails application from using the database while mysqldump was running and it was necessary to use the parameter "--single-transaction" like so:<br />
<br />
<blockquote>mysqldump -u myuser -pmypass --single-transaction db_name > output.sql</blockquote><br />
If you have any tips to share, I welcome them in the comments.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-2162770374403066826.post-6773639207476374842010-10-13T19:49:00.000-07:002010-10-13T19:49:17.608-07:00To Freemium or Not to Freemium?<div class="separator" style="clear: both; text-align: center;"><a href="http://ws.amazon.com/widgets/q?MarketPlace=US&ServiceVersion=20070822&ID=AsinImage&WS=1&Format=_SL160_&ASIN=B00342VEP6&tag=bulldogbreeds-20" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img alt="Free: The Future of a Radical Price" border="0" src="http://ws.amazon.com/widgets/q?MarketPlace=US&ServiceVersion=20070822&ID=AsinImage&WS=1&Format=_SL160_&ASIN=B00342VEP6&tag=bulldogbreeds-20" /></a></div>This is a topic that comes up a lot when dealing with Internet businesses. I would say that the topic is interesting, but I actually find it somewhat nauseating as I reflect on the number of online quarrels I've encountered when reading about the Freemium debate. This is why I found it refreshing to come across <a href="http://www.mailchimp.com/blog/going-freemium-one-year-later/">this nice article</a> from the folks at MailChimp. A beautifully laid out case study on some of the pros and cons of Freemium, and an encouragement to startups that have been around a little too long to be considered a true startup. Personally, the idea of Freemium appeals to me as it gives users a chance to taste your product for free, and gives you (the business) an opportunity to learn from your users and grow your user base. So, "To Freemium or Not to Freemium?" -- you decide.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-2162770374403066826.post-70636419340938205672010-07-01T14:15:00.000-07:002010-07-01T14:15:57.614-07:00PayPal Charset IPN IssueThis post is for anyone experiencing problems with the PayPal verification step in the IPN process. After pulling my hair out trying to figure this out, I found the <a href="http://www.blogger.com/">solution on another blog</a>.<br />
<br />
The Problem:<br />
PayPal by default posts its IPNs using the Windows-1252 character encoding (why would they do that?). If you are like any sensible UTF-8-loving developer, then you will find that your postbacks to PayPal are not receiving the VERIFIED status that ensures they are a legitimate IPN. It seems non-ASCII characters in the IPN parameters are not being interpreted correctly by PayPal (because they are expecting Windows-1252 and you are sending UTF-8).<br />
<br />
The Fix:<br />
A simple setting in your business PayPal account under "Language Encodings". You will need to select "More Options" to find the screen that allows you to select UTF-8 from the dropdown.Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-2162770374403066826.post-36022672641042623502010-06-15T13:46:00.000-07:002010-06-15T13:46:38.226-07:00Beware of YSlow and GZip<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2oZcdkYf9d0oyDDoN9XFKzBGuMjkpK9xOfL-owg5xDTDqlFG4G-CMtOssPB1Hi4MrqjwpVGxx2eiMrP_R9RW_hmtiboVpxEzclHeoBjLxR67SpTk9gr9sxabj9kvKa1t3PRrdvZcb-14/s1600/compress.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2oZcdkYf9d0oyDDoN9XFKzBGuMjkpK9xOfL-owg5xDTDqlFG4G-CMtOssPB1Hi4MrqjwpVGxx2eiMrP_R9RW_hmtiboVpxEzclHeoBjLxR67SpTk9gr9sxabj9kvKa1t3PRrdvZcb-14/s320/compress.jpg" /></a></div><br />
Much time can be wasted trying to configure mod_deflate IF it is already properly configured. Although I had previously enabled compression using this <a href="http://httpd.apache.org/docs/2.0/mod/mod_deflate.html">simple and powerful Apache module</a>, YSlow was giving me an F for not compressing the page text as well as js and css files. To make a short story even shorter I will simply say this: <br />
<br />
YSlow does not accurately detect gzip compression. If you are uncertain, check the response headers for "Vary: Accept Encoding" or use <a href="http://www.port80software.com/support/p80tools.asp">Port80's Compression Check</a>.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-2162770374403066826.post-74599336500330433232010-06-01T13:31:00.000-07:002012-06-27T11:12:31.228-07:00Ruby Part-of-Speech Tagger Shootout<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWdJmv2xSItwt4rDUNfC9ooDv4AUezITzLouVU_T46JAOxDYfEKT_xTFGT0LLXrHiWD1jojlFippgWli7FhPkQKxqXWqHzL-T2ZwPZp6nTz5XOTiJikci5xja9mBU7uOEmHOhlTMTbATw/s1600/tag.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWdJmv2xSItwt4rDUNfC9ooDv4AUezITzLouVU_T46JAOxDYfEKT_xTFGT0LLXrHiWD1jojlFippgWli7FhPkQKxqXWqHzL-T2ZwPZp6nTz5XOTiJikci5xja9mBU7uOEmHOhlTMTbATw/s320/tag.jpg" /></a></div>
An accurate and efficient Part of Speech Tagger represents a valuable tool for various areas of natural language processing. I use POS Tagging as a means of detecting invalid text, but there are many other possible uses as well. Regardless of how you are using a POS Tagger, you may find this benchmark of two Ruby POS Tagging libraries helpful.<br />
<br />
<br />
<br />
<b>The Players</b><br />
There are several taggers available, but I settled on testing the following two that are available as gems and seemingly robust:<br />
<ol>
<li><a href="http://engtagger.rubyforge.org/"><b><span style="color: #274e13;">EngTagger</span></b>:</a> a corpus-trained, probabilistic tagger (port of Perl Lingua::EN::Tagger)</li>
<li><a href="http://github.com/taf2/rb-brill-tagger"><b><span style="color: #274e13;">RubyTagger</span></b></a> (rb-brill-tagger): a rule based tagger</li>
</ol>
<a href="http://www.amazon.com/Foundations-Statistical-Natural-Language-Processing/dp/0262133601?ie=UTF8&tag=bulldogbreeds-20&link_code=bil&camp=213689&creative=392969" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;" target="_blank"><img alt="Foundations of Statistical Natural Language Processing" src="http://ws.amazon.com/widgets/q?MarketPlace=US&ServiceVersion=20070822&ID=AsinImage&WS=1&Format=_SL160_&ASIN=0262133601&tag=bulldogbreeds-20" /></a>Some of the ones excluded that you may be interested in considering:<img alt="" border="0" height="1" src="http://www.assoc-amazon.com/e/ir?t=bulldogbreeds-20&l=bil&camp=213689&creative=392969&o=1&a=0262133601" style="border: medium none ! important; margin: 0px ! important; padding: 0px ! important;" width="1" /><br />
<ul>
<li><a href="http://www.markwatson.com/opensource/">Mark Watson's Tagger</a></li>
<li><a href="http://chasen.org/%7Etaku/software/yamcha/">YamCha</a> - Yet Another Multipurpose CHunk Analyzer</li>
</ul>
Both EngTagger and RubyTagger provide a simple API and could be easily installed as gems. (NOTE: I believe the RubyTagger gem has a C dependency). In order to get a fairly accurate idea of performance, the gems were tested under 4 scenarios:<br />
<br />
1-A: 10 times create an instance of the tagger and tag a <i>short</i> piece of text<br />
1-B: 10 times create an instance of the tagger and tag a <i>long</i> piece of text<br />
2-A: create an instance of the tagger once and tag a <i>short</i> piece of text 10 times<br />
2-B: create an instance of the tagger once and tag a <i>long</i> piece of text 10 times<br />
<br />
Before looking at the results, let's examine the main portion of the benchmark.rb file:<br />
<pre class="brush:ruby">
# Scenario 1-A: load the tagger each time before processing text; use SHORT text for tagging
Benchmark.bmbm do |b|
b.report("1-A: eng tagger") {10.times { engtagger = EngTagger.new; engtagger.get_readable(SHORT_TEXT).split(' ').collect{|x| x.split('/')} }}
b.report("1-A: rb tagger") {10.times { rbtagger = Brill::Tagger.new; rbtagger.tag(SHORT_TEXT) } }
end
# Scenario 1-B: load the tagger each time before processing text; use LONG text for tagging
Benchmark.bmbm do |b|
b.report("1-B: eng tagger") {10.times { engtagger = EngTagger.new; engtagger.get_readable(LONG_TEXT).split(' ').collect{|x| x.split('/')} }}
b.report("1-B: rb tagger") {10.times { rbtagger = Brill::Tagger.new; rbtagger.tag(LONG_TEXT) } }
end
# Scenario 2-A: load the tagger once, then process SHORT text
Benchmark.bmbm do |b|
b.report("2-A: eng tagger") { engtagger = EngTagger.new; 10.times { engtagger.get_readable(SHORT_TEXT).split(' ').collect{|x| x.split('/')} }}
b.report("2-A: rb tagger") {rbtagger = Brill::Tagger.new; 10.times { rbtagger.tag(SHORT_TEXT) } }
end
# Scenario 2-B: load the tagger once, then process LONG text
Benchmark.bmbm do |b|
b.report("2-B: eng tagger") { engtagger = EngTagger.new; 10.times { engtagger.get_readable(LONG_TEXT).split(' ').collect{|x| x.split('/')} }}
b.report("2-B: rb tagger") {rbtagger = Brill::Tagger.new; 10.times { rbtagger.tag(LONG_TEXT) } }
end
</pre>
<br />
You may notice the extra processing of the EngTagger output:<br />
<span style="font-size: x-small;"><span style="background-color: #fff2cc;">engtagger.get_readable(SHORT_TEXT).split(' ').collect{|x| x.split('/')}</span></span><br />
This is done so that the output matches that of the RubyTagger, which is an array of 2-item arrays.<br />
<br />
<b>The Results</b><br />
The output of the benchmark script is below (and slightly tidied up).<br />
<blockquote>
<span style="font-size: x-small;"><br />
</span><br />
<span style="font-size: small;">Scenario 1-A: load the tagger each time before processing SHORT text<br />
user system total real<br />
1-A: eng tagger 9.340000 0.040000 9.380000 ( 9.652550)<br />
1-A: rb tagger 22.310000 2.180000 24.490000 ( 25.109431)<br />
<br />
<br />
Scenario 1-B: load the tagger each time before processing LONG text<br />
user system total real<br />
1-B: eng tagger 11.880000 0.710000 12.590000 ( 12.940737)<br />
1-B: rb tagger 23.330000 2.350000 25.680000 ( 26.337501)<br />
<br />
Scenario 2-A: load the tagger once, then process SHORT text 10 times<br />
user system total real<br />
2-A: eng tagger 0.600000 0.000000 0.600000 ( 0.652037)<br />
2-A: rb tagger 1.950000 0.240000 2.190000 ( 2.252128)<br />
<br />
<br />
Scenario 2-B: load the tagger once, then process LONG text 10 times<br />
user system total real<br />
2-B: eng tagger 2.500000 0.260000 2.760000 ( 2.840162)<br />
2-B: rb tagger 2.710000 0.250000 2.960000 ( 3.048174)</span></blockquote>
It appears that the EngTagger outperforms the RubyTagger in all tests BUT what happens if we change the number of iterations to 50 (rather than 10). After all, 10 times is rather small:<br />
<br />
<blockquote>
Scenario 1-A: load the tagger each time before processing SHORT text<br />
user system total real<br />
1-A: eng tagger 45.370000 0.310000 45.680000 ( 46.750664)<br />
1-A: rb tagger 117.960000 11.450000 129.410000 (132.715563)<br />
<br />
<br />
Scenario 1-B: load the tagger each time before processing LONG text<br />
user system total real<br />
1-B: eng tagger 57.110000 3.610000 60.720000 ( 62.123540)<br />
1-B: rb tagger 121.770000 11.050000 132.820000 (136.239764)<br />
<br />
<br />
Scenario 2-A: load the tagger once, then process SHORT text 50.times<br />
user system total real<br />
2-A: eng tagger 0.790000 0.060000 0.850000 ( 0.896051)<br />
2-A: rb tagger 2.040000 0.230000 2.270000 ( 2.340134)<br />
<br />
<br />
Scenario 2-B: load the tagger once, then process LONG text 50.times<br />
user system total real<br />
2-B: eng tagger 9.340000 0.850000 10.190000 ( 10.528600)<br />
2-B: rb tagger 6.000000 0.470000 6.470000 ( 6.636378)</blockquote>
With 50 iterations, the EngTagger outperforms RubyTagger on all but the last task. Interestingly, the last scenario is the one that most closely matches the real-world application being built.<br />
<br />
<b>Lessons Learned</b><br />
<ol>
<li>EngTagger and RubyTagger perform optimally under different conditions</li>
<li>Benchmarks should mimic your application's usage as closely as possible</li>
</ol>Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-2162770374403066826.post-9969902878663179882010-05-25T14:10:00.000-07:002012-06-27T11:00:19.547-07:00Statistical Analysis using Ruby<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPjTbOW-EcqUIgDXR7tn48H1FYx84hCICfwHRDID5nuU9KBKgJzI-yuUVKqZoLagJSoxidlHjTFeAbGkq61NM2tj4C7eqxuV3ftsD4Cu1KqLezgkTvu5ZDrqQ_Ybr6HiYNRnArZFLdUmM/s1600/statistics.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPjTbOW-EcqUIgDXR7tn48H1FYx84hCICfwHRDID5nuU9KBKgJzI-yuUVKqZoLagJSoxidlHjTFeAbGkq61NM2tj4C7eqxuV3ftsD4Cu1KqLezgkTvu5ZDrqQ_Ybr6HiYNRnArZFLdUmM/s320/statistics.jpg" width="320" /></a></div>
Having migrated to Ruby from a Java background, I sometimes find myself longing for the vast and robust libraries that exist in the Java ecosystem. While preparing for a natural language processing task, I considered revisiting the Weka software that I had used in the past for machine learning and statistical analysis. But, it sure would be nice if something existed in Ruby for this sort of work. Enter <a href="http://ruby-statsample.rubyforge.org/">Statsample</a>.<br />
<br />
<br />
While not a machine learning package, this statistics library utilizes the Gnu Scientific Library to provide me with the two features upon which this task depended: Multiple Regression and Pearson r Correlation Coefficient. Here are some examples of both at work:<br />
<br />
<pre class="brush:ruby">
# Calculate correlation coefficient
b=(1..100).collect { rand(100)}.to_scale
Statsample::Bivariate.pearson(a,b)
# Multiple Regression
a=1000.times.collect {rand}.to_scale
b=1000.times.collect {rand}.to_scale
c=1000.times.collect {rand}.to_scale
ds={'a'=>a,'b'=>b,'c'=>c}.to_dataset
ds['y']=ds.collect{|row| row['a']*5+row['b']*3+row['c']*2+rand()}
lr=Statsample::Regression.multiple(ds,'y')
puts lr.summary
Summary for regression of a,b,c over y
*************************************************************
Engine: Statsample::Regression::Multiple::AlglibEngine
Cases(listwise)=1000(1000)
r=0.986
r2=0.973
Equation=0.504+5.011a + 2.995b + 1.988c
</pre>
Source: http://ruby-statsample.rubyforge.org/Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-2162770374403066826.post-38547972935691429912010-03-17T07:45:00.000-07:002010-03-17T07:45:59.831-07:00Play Video Games and Save the World<a href="http://www.amazon.com/Call-Duty-Modern-Warfare-2/dp/B00269QLI8?ie=UTF8&tag=bulldogbreeds-20&link_code=bil&camp=213689&creative=392969" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;" target="_blank"><img alt="Call of Duty: Modern Warfare 2" src="http://ws.amazon.com/widgets/q?MarketPlace=US&ServiceVersion=20070822&ID=AsinImage&WS=1&Format=_SL160_&ASIN=B00269QLI8&tag=bulldogbreeds-20" /></a><br />
I don't have as much time for video games as I onc<img alt="" border="0" height="1" src="http://www.assoc-amazon.com/e/ir?t=bulldogbreeds-20&l=bil&camp=213689&creative=392969&o=1&a=B00269QLI8" style="border: medium none ! important; margin: 0px ! important; padding: 0px ! important;" width="1" />e did, but it's nice to hear that v<a href="http://www.ted.com/talks/jane_mcgonigal_gaming_can_make_a_better_world.html">ideo games will solve the world's problems</a>. This video from TED.com points to additional emerging research that video games do not rot the mind as once thought:<br />
<br />
<a href="http://www.ted.com/talks/jane_mcgonigal_gaming_can_make_a_better_world.html">http://www.ted.com/talks/jane_mcgonigal_gaming_can_make_a_better_world.html</a>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-2162770374403066826.post-87194400831610073672010-03-08T11:01:00.000-08:002010-03-08T11:02:14.889-08:00Controlling Web-Bots with CurbIt<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiw8zIStnTAER1ww7wz9gW36aqR_U0jwkN0Jpn_NCXx10DAu-yfG9n7gKeH4XkD7NLM5oe_U9PMpIvN929A9YqS6ROjc7thaNc2zlGZVrZ9X2S8gmlQJt23ZBKEp80eXcMP6bdCNyu8rgU/s1600-h/bot.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiw8zIStnTAER1ww7wz9gW36aqR_U0jwkN0Jpn_NCXx10DAu-yfG9n7gKeH4XkD7NLM5oe_U9PMpIvN929A9YqS6ROjc7thaNc2zlGZVrZ9X2S8gmlQJt23ZBKEp80eXcMP6bdCNyu8rgU/s200/bot.jpg" width="126" /></a></div>Whether you have a content-heavy site or a very application-centric website, bots and harvesters can wreak havoc by eating up CPU cycles, memory, and system resources. These ubiquitous pests will gladly retrieve all of your site's content with little regard for copyright laws or your terms of service.<br />
<br />
One helpful tool for limiting many types of harmful bots and crawlers is a Ruby gem or plugin called CurbIt. CurbIt adds application level rate limiting to your Rails app. I recently had the pleasure of utilizing CurbIt on the <a href="http://www.paperrater.com/">Paper Rater</a> website to limit the number of submissions. This helps us to ensure that humans are submitting documents, but without bothering our users with a CAPTCHA.<br />
<br />
Example usage from the <a href="http://github.com/ssayles/curbit">CurbIt github page</a>:<br />
<br />
<code></code><br />
<pre>class InvitesController < ApplicationController
def invite
# invite logic...
end
rate_limit :invite, :max_calls => 2, :time_limit => 30.seconds, :wait_time => 1.minute
end
</pre>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-2162770374403066826.post-82552829670312296752010-02-18T06:54:00.000-08:002010-03-08T04:20:32.591-08:00Should You Move to the Cloud?<a href="http://www.amazon.com/Cloud-Computing-Dummies-Judith-Hurwitz/dp/0470484705?ie=UTF8&tag=bulldogbreeds-20&link_code=bil&camp=213689&creative=392969" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;" target="_blank"><img alt="Cloud Computing For Dummies" src="http://ws.amazon.com/widgets/q?MarketPlace=US&ServiceVersion=20070822&ID=AsinImage&WS=1&Format=_SL160_&ASIN=0470484705&tag=bulldogbreeds-20" /></a><img alt="" border="0" height="1" src="http://www.assoc-amazon.com/e/ir?t=bulldogbreeds-20&l=bil&camp=213689&creative=392969&o=1&a=0470484705" style="border: medium none ! important; margin: 0px ! important; padding: 0px ! important;" width="1" />At Oasic we host a lot of websites and one of the most boring tasks involves setting up new servers, either for a new website or as part of a migration/expansion. It's one of my least favorite parts of the job. With many of the cloud offerings out there (Heroku, Google AppEngine, EngineYard, etc.), I've begun to dip my toes in the water. But why not just dive in?<br />
<br />
<b>Most Apps are NOT Designed for the Cloud</b><br />
<br />
The concept of cloud hosting, along with its advantages and restrictions, is a relatively new phenomenon and most application frameworks, libraries, processes, and tools are not developed with the cloud in mind. Heroku's simple restriction of a read-only filesystem for all but a couple directories, means that you have to work around this. Restrictions on background processes means that you need to work with their related add-on, or come up with another solution. It begins to feel like deployment is a series of workarounds, and I'm not certain that's any better than setting up a new server.<br />
<br />
<b>Middle Ground</b><br />
<br />
VPS providers like Amazon EC2, Slicehost, and WebbyNode provide a middle ground between a full cloud solution and hosting on dedicated servers. They allow instances to be setup more easily, scaled up and down, and backed up at nice prices and good uptime. And, they allow full access to the server. You can write to the filesystems, access databases directly, run background tasks, poke around on the server via SSH, create shell scripts, etc.<br />
<br />
<b>Further Reading</b><br />
<br />
Moving to a full cloud stack requires a loss of flexibility that many businesses can't allow. Here is a good case study about GitHub's move out of the cloud:<br />
<br />
<a href="http://github.com/blog/493-github-is-moving-to-rackspace">http://github.com/blog/493-github-is-moving-to-rackspace</a><br />
<br />
And, of course, there are many businesses that benefit from the advantages of moving the cloud: inherit load-balancing, fault-tolerance, pay-per-usage, ease of deployment, etc.<br />
<br />
So, review your options, consider your needs, and make the best choice for your application's hosting.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-2162770374403066826.post-30587841904389578892010-02-16T07:58:00.000-08:002010-03-08T05:38:06.094-08:00Blogger Beats Typo<a href="http://www.amazon.com/Blogger-Customize-templates-advertising-Technologies/dp/184719317X?ie=UTF8&tag=bulldogbreeds-20&link_code=bil&camp=213689&creative=392969" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;" target="_blank"><img alt="Blogger: Beyond the Basics: Customize and promote your blog with original templates, analytics, advertising, and SEO (From Technologies to Solutions)" src="http://ws.amazon.com/widgets/q?MarketPlace=US&ServiceVersion=20070822&ID=AsinImage&WS=1&Format=_SL160_&ASIN=184719317X&tag=bulldogbreeds-20" /></a><img alt="" border="0" height="1" src="http://www.assoc-amazon.com/e/ir?t=bulldogbreeds-20&l=bil&camp=213689&creative=392969&o=1&a=184719317X" style="border: medium none ! important; margin: 0px ! important; padding: 0px ! important;" width="1" />So I set out to create a blog for my website and spent some considerable time looking at possible options. Most of my development is done in Ruby these days, so I wanted to stick with a Ruby package if possible. These were my finalists:<br />
<ul><li>Radiant CMS: not a pure blogging solution, but a full-featured, extensible CMS with blogging modules. It would allow me to add new functionality to my programmer-heart's delight.</li>
<li>Typo: the oldest, tried-and-true Ruby blogging system out there. This heavyweight is a non-nonsense blogging machine.</li>
<li>Mephisto: a leaner blogging app than Typo, but still with nice features.</li>
</ul><b>And the winner is...</b><br />
<br />
Typo -- at least it seemed. I used the typo gem to install my blog. Added database permissions and began configuring things. I edited the default Hello World post that was auto-created. I was on my way until I created my first new post. No visual errors appeared, but the posts were not being saved. What could be going on?<br />
<br />
I often enjoy a good troubleshooting session, but not this time around. I had 3 blogs needing to be setup (for various sites) and I didn't want to deal with these issues. I decided to try out Google's hosted Blogger and the rest is history!<br />
<br />
Although there is no Markdown, Textile, or other markup suppport, it does include a WYSIWYG editor with full html tags. Here's some other benefits:<br />
<ul><li>use your own domain for free (or subdomain as I have done with this blog)</li>
<li>fully customizable layout</li>
<li>API for programmatic access</li>
<li>3 minute setup of a new blog</li>
<li>Google handles everything for you (1 less component to manage)</li>
<li>did I mention it was free?</li>
</ul>Unknownnoreply@blogger.com0