Imdb api
I have created a Ruby code that gathers information from IMDB and presents it in a nice class. The code uses why's hpricot v0.4.76.
Creating an object. The HTML is only loaded once as long as you have the object.
If using the IMDB class in a command line, the menu option will present you with the titles and years that are listed on the IMDB search site and you enter a number. If there is menu (default value is false) it will pick the first link found. If no links are found it raises an error
Now lets say you want to print the list of actors.
Although most movies do not have a plot IMDB still tries to list one.
So far its not to hard. BUT BECKER!! I want the poster!
Have no FEAR (some what)!
This will return the a img html tag, I am working on making this better.
Some other things.
IMDB has no published API. I have found no other sites providing a good API. So I made one.
http://stephenbeckeriv.com/movie/office_space
The title uses _ for spaces. You can also use %20 but I made it easy. You will have uri encode your titles.
Also if you are not getting the search result you want, try looking at the returning titles.
http://stephenbeckeriv.com/movie_titles/office_space
Please email me with comments and suggestions!
Creating an object. The HTML is only loaded once as long as you have the object.
movie=IMDB.new("monty python holy grail") or menu=true movie=IMDB.new("monty python holy grail",menu)
If using the IMDB class in a command line, the menu option will present you with the titles and years that are listed on the IMDB search site and you enter a number. If there is menu (default value is false) it will pick the first link found. If no links are found it raises an error
Now lets say you want to print the list of actors.
.actors is a hash where the key is he actor's name and the hash is the list of roles(CVS).movie.actors.each{|name,role| p name+" starred in "+ movie.title+" as "+role+"\n" }
Although most movies do not have a plot IMDB still tries to list one.
p "This movies plot might be like: "+movie.plot
So far its not to hard. BUT BECKER!! I want the poster!
Have no FEAR (some what)!
movie.poster
This will return the a img html tag, I am working on making this better.
Some other things.
movie.director movie.run_time movie.year movie.genres #an array movie.mpaa #html with rating and reason movie.to_xml#ohh why would I do this?
IMDB has no published API. I have found no other sites providing a good API. So I made one.
http://stephenbeckeriv.com/movie/office_space
The title uses _ for spaces. You can also use %20 but I made it easy. You will have uri encode your titles.
Also if you are not getting the search result you want, try looking at the returning titles.
http://stephenbeckeriv.com/movie_titles/office_space
Please email me with comments and suggestions!
Site map
Looking around google's Webmaster Tools I found a well hidden link to google's help page about site map formatting. Looking at the Sitemap protocol (just a simple xml file) I set off to create a method that would take my dynamic website and create a nice little map for it. After all if you can not find it in google, you really don't need it.
I am sure I can come up with a general method to do this, but currently it is specific to how I have created my website.
You have to start your xml docuemnt. I really like using rexml/document, you will also need zlib class
And you are done! Make this a method on your admin console and submit the link to google. A few things I want to do with this is provide a link to the xml, and create a dynamic site map with ajax using the same ideas. If someone wanted to present a map to my site they could use the xml and format it how they like. I am thinking about creating a general map that looks at the views and routes.rb to try and create the proper links.
Update:
Turns out when i added my site to the webmaster tools in google i used http://stephenbeckeriv.com/ not http://www.stephenbeckeriv.com/ which it does not like my site map because I added the www. How fun and flexible.
I am sure I can come up with a general method to do this, but currently it is specific to how I have created my website.
You have to start your xml docuemnt. I really like using rexml/document, you will also need zlib class
data=REXML::Document.new('<?xml version="1.0" encoding="UTF-8"?>') base=data.add_element("urlset") base.attributes["xmlns"]="http://www.google.com/schemas/sitemap/0.84"</pre> <br>These values are found on the Sitemap protocol website<br><br> Then I add in my static content values<br><pre> el=base.add_element("url") el.add_element("loc").text="http://www.stephenbeckeriv.com/" #Time needs to be in w3c formatting, For the life of me I can not use Time's w3cdtf el.add_element("lastmod").text=Time.now.strftime("%Y-%m-%d") el.add_element("changefreq").text="weekly" el.add_element("priority").text="0.5" #turns out they do not like svn sites #el=base.add_element("url") #el.add_element("loc").text="http://www.svn.stephenbeckeriv.com/code" #el.add_element("lastmod").text=Time.now.strftime("%Y-%m-%d") #el.add_element("changefreq").text="weekly" #el.add_element("priority").text="0.5"</pre> <BR><BR>Then I add in the context part of my site<br> <pre> @context_list = Post.find(:all,:select=>"context") arr=[] @context_list.each{|x|arr.push(x.context)} arr.uniq.each{|context| el=base.add_element("url") el.add_element("loc").text="http://www.stephenbeckeriv.com/#{context}" el.add_element("lastmod").text=Time.now.strftime("%Y-%m-%d") el.add_element("changefreq").text="weekly" el.add_element("priority").text="0.5" } </pre> <br> I know I can uniq my results with the sql, but I could not remember the code for this. <br><BR> Then I add my dynamic content<br><pre> @post=Post.find(:all) @post.each{|post| el=base.add_element("url") #if you read my first post you know why this works el.add_element("loc").text="http://www.stephenbeckeriv.com/#{URI.encode(post.title.gsub(" ","_"))}" a=Time.parse("#{post.created}")#it did not like having the date formatting on one line. el.add_element("lastmod").text=a.strftime("%Y-%m-%d") el.add_element("changefreq").text="never" el.add_element("priority").text="0.8" }</pre> <br><BR>We Now have all the web pages for my site. Now we need to save this to a file in public. There are some limits for the file. You can not have over 500,000 urls, the file must be less then 10MB. If this is a problem, you have to use the site map index files. I do not cover this. <br><br><pre> result="" data.write(result) dir=Dir.pwd #local dir.does not have /public dir<< "/public/sitemap.xml.gz" if !dir.include("/public") #server does have public dir<< "/sitemap.xml.gz" if dir.include("/public") Zlib::GzipWriter.open(Dir.pwd+"/public/sitemap.xml.gz"){|file| file.write result }
And you are done! Make this a method on your admin console and submit the link to google. A few things I want to do with this is provide a link to the xml, and create a dynamic site map with ajax using the same ideas. If someone wanted to present a map to my site they could use the xml and format it how they like. I am thinking about creating a general map that looks at the views and routes.rb to try and create the proper links.
Update:
Turns out when i added my site to the webmaster tools in google i used http://stephenbeckeriv.com/ not http://www.stephenbeckeriv.com/ which it does not like my site map because I added the www. How fun and flexible.
Prepared statements
I have set out to create prepared statement support for active record. So far it is looking good. I am looking at different ways to manage the statements. The biggest pain will be supporting the syntax of different databases. I will start out with statements in Postgres.
Rails conf 2007
It's official I will be speaking at Rails Conf 2007!
Rails conf bio
Stephen Becker IV is a Cancer with a a smooth demeanor and a voice like velour. His turn-ons are long walks on the beach, futurama and all night caffeine induced bing programming sessions. His turn-offs are stupidity and the lack of velour clothing currently in stores. Presently working for Vonage as a rails developer in New Jersey.
Rails Conf link
Rails Conf link
Prepared statements start
Using prepared statements in your applications is like giving a tug on your spice weasel!
I am not going to talk about why prepare statements are important. That information can be found at google, or from years of doing database work. This is how I am going add support for them in to active record.
The syntax is not complete yet but it will go down something like this...
I think its simple and sweet. We are also working on "auto_magic" mode for people who do not know sql. This will high jack active records processes of creating the sql, and create statements, manage them and run them instead of the normal sql statements. I just want to make sure no one can say I discriminate against people who do not know sql. Also, I have yet to test what happens if you call the a statement with the wrong model. Currently we do not have the concept of this model owns this statement. We need a better handling of errors and more testing with Oracle. But as Nick always says "Its piratically done"
I am not going to talk about why prepare statements are important. That information can be found at google, or from years of doing database work. This is how I am going add support for them in to active record.
The syntax is not complete yet but it will go down something like this...
ActiveRecord::Base.create_ps({:key=>'Pstatemnet1', :sql=>'select * from cost_to_cost where id=$1;}) </code <br> <br><br>Notice the use of very helpful names. Also, notice the ps and not prepared_statement the less I have to type the more I will use it, and it will just be an ailis anyway. So, key is going to be the name that is associated with the statement. As I currently understand it postgres allows you to set this and oracle slaps you in the face. The the sql statement will be more like active record and use ? for the value locations. The syntax for this statement comes from postgres. <br><br> Yah we made a statement.. but can we use it?<BR><br><br> <code> SpaceGhost.find(:ps,:key=>"Pstatement1",:conditions=>[100]) <br> SpaceGhost.find_by_ps(:key=>"Pstatement1",:conditions=>[15])<br>
I think its simple and sweet. We are also working on "auto_magic" mode for people who do not know sql. This will high jack active records processes of creating the sql, and create statements, manage them and run them instead of the normal sql statements. I just want to make sure no one can say I discriminate against people who do not know sql. Also, I have yet to test what happens if you call the a statement with the wrong model. Currently we do not have the concept of this model owns this statement. We need a better handling of errors and more testing with Oracle. But as Nick always says "Its piratically done"
Komodo 4
I use Komodo IDE at work and at home I would always use Rad Rails. That was the case until I started running 64bit suse. Rad Rails will not work for me, and it is not worth my effort to figure it out. Also Komodo just released 4.0!
I was sad at first because I did not see a price for a personal edition. So I am just the demo version of 4.0 and I found they have a free version which they call Komodo Edit. Found here ! I have yet to install the free version but if its close enough to the pay version I am set.
There is a big list some where of the features for komodo but there are only a few things that draw me to this product. Number 1 with out a doubt is the toolbox. I store bits of repeatable code and make them hot keys. This makes writing unit test, change logs, formatted rdocs comments so easy. I have also created a package for the tool box that includes the common RoR actions like rad rails has. That can be downloaded from here. Number 2 is the tab complete. In 4.0 you have to turn on the tab complete because its bound with different keys. I think they did a good job at guessing which word I want. They have an open that strips the whitspaces at the end of lines which includes those killer windows ^M's.
It is missing a few things that rad rails has. One is the hot key to take you from the controller to the view and back again. The auto formatting for indentation is very nice because sometimes i get out of control.
I was sad at first because I did not see a price for a personal edition. So I am just the demo version of 4.0 and I found they have a free version which they call Komodo Edit. Found here ! I have yet to install the free version but if its close enough to the pay version I am set.
There is a big list some where of the features for komodo but there are only a few things that draw me to this product. Number 1 with out a doubt is the toolbox. I store bits of repeatable code and make them hot keys. This makes writing unit test, change logs, formatted rdocs comments so easy. I have also created a package for the tool box that includes the common RoR actions like rad rails has. That can be downloaded from here. Number 2 is the tab complete. In 4.0 you have to turn on the tab complete because its bound with different keys. I think they did a good job at guessing which word I want. They have an open that strips the whitspaces at the end of lines which includes those killer windows ^M's.
It is missing a few things that rad rails has. One is the hot key to take you from the controller to the view and back again. The auto formatting for indentation is very nice because sometimes i get out of control.
Ruby should
Ruby should have a ternary operator for <=>
Ex:
string would be eqaul to "is less".. I will start working on this..
Ex:
string = 1<=>2 ? "is less":"are equal":"is greater"
string would be eqaul to "is less".. I will start working on this..
Gem to rails plugin
I most recently wrote a class to interface with IMBD.com. This class requires a library called hpricot. I locally gem install hpricot and create my code. I then added my code to my website. Do to the fact that I use shared hosting I do not have the proper rights to gem install my needed libraries. I could open a ticket with my hosting company, and wait for someone to get to it. Nah, I don't want to do that. I decided to look in to creating a plugin from a gem.
As it turns out most of the time you can.
This what I did to create a plugin for hpricot
I tried to do this with rmagik but it did not work. So its not 100% but it can be done. I am working on a rake command, much like rake rails, to create plugins from gems.
As it turns out most of the time you can.
This what I did to create a plugin for hpricot
1)Make rails project
2)ruby script/gen plugin hpricot
3)Gem install
a)If you can on the server change the gem root to a
local file you have RW access to and gem install hpricot -i /path/you/can/write
b)If you can not gem install on the server do it locally,but
do not cross windows with linux (different flavors do not seem to matter
to much)
4) Locate the installed gem package
5) Copy lib and ext from the gem to ror/vendor/plugins/hpricot/
6) Should work like a champ.
I tried to do this with rmagik but it did not work. So its not 100% but it can be done. I am working on a rake command, much like rake rails, to create plugins from gems.
Us websites
I found a list of words that end in us. I do not know how complete my list was but thats not the point. I made a script to find 3,4 and 5 letter words that I could make sites out of and a list of words that I could create with sub-domains. Just like script.aculo.us For example hero.us (taken) would give me the words [lecherous , ocherous , treacherous]. The longest set of words I have is for ero.us (taken), cio.us (take). oro.us(taken), geo.us (taken)
I can not find a 3 letter .us that is not taken. Well not one with many words.
So it looks like I am very late to this party. I added the text file to my svn http://svn.stephenbeckeriv.com/code/other/ussites.txt
I can not find a 3 letter .us that is not taken. Well not one with many words.
So it looks like I am very late to this party. I added the text file to my svn http://svn.stephenbeckeriv.com/code/other/ussites.txt
Segfault4r
I just released segfault4r. I created a rubyforge page,raa page and made an announcement to ruby talk. I can only hope the lord and master of ruby can forgive me for my sin.
Example:
Example:
require "rubygems" require "segfault" Segfault.now puts "Forgive me matz."
Rails conf day 3
Another great day at rails conf 2007! Morgan did his talk to day and it went well. The Extra Action Marching Band stopped by for a quick demo. I just got back from the bar they were playing in and it was amazingly fun. I talked to lots of people today google recruiters, tyler from amazon, some people working with nasa, and other people. I am excited to host my own presentation. I have to have someone with a Mac convert it in to a PDF first. I will post that link when I have it. I have found 2 blogs about my talk and one on morgans (all here). In the blog about morgan's talk they called us .net developers. That cuts me deep. I have never been a .net programmer. I have done (in order of first appearance) C,Ruby,Python,and a little Java. After checking out the blogs make sure to look on flickr. I am looking forward to tomorrow's talks.
Starting the end
Looks like 100% chance of rain.. I wanted to talk about the speeches I saw yesterday. I went to two talks on virtualization which I liked because I run VMs. I am glad to see that having a server in a VM only has a performance reduction of 5-8% I will be looking in to creating ruby scripts to create and manage Xen images. Another talk was about creating scriptaculous effects. I did want to see more of the graphic engine and that kinda logic but it was very informative. One example that was shown was blinding left to right or right to left. I want to add this effect to the calender on my page and start to create more effects. I am surprised some one does not have the star wars text scrolling effect. It would be fun to do but I think i will start out a little easier. Time to get some food and walk in the rain.
End of the end
It was a lot of fun. I have not posted about the talks because you can get the slides yourself. I will be posting what I create because of the talks I have seen. I am just glad that I could speak in front of so many of my peers.
Ruby formatter
Everyone has a preference on formatting code. I have found that there are a lot of very adamant people when it comes to the formatting of code. Unless a developer is using a language, like Python, that requires you to indent code it is up to the developer to correctly indent. When working with a team of people of you can have someone who does not care as much for strict formatting and you can have some one who is a formatting extremist.
To resolve some issues with other team members I have created a simple ruby formatter. I call it simple because it only formats one line at a time. There is not looking around at different lines, no knowledge of location in an object, or even scoping. I would like to create a rbchecker that would be like Pychecker, but this project is not that advanced. There is no parsing of ruby syntax, instead I am just looking for simple patterns. I am sure I missed a pattern or two but I have the option to create a back of the original file. I suggest turning on the back up for the first time you run this.
After I finished up a few test i figured i would look around to see if there were any other formatters. Turns out there are a few but, sine I am already done... Why not release it anyways?
Right now it is mostly comments, and no i did not run the formatter on it's self. I have been testing it on other projects of mine and koders.com ruby files.
Where to get it:
My svn http://svn.stephenbeckeriv.com/code/ruby_formatter/
How to use it:
Required:
-f file/location/e.rb
This can also be called with commas
-f file/location/e.rb,file/location/q.rb,file/location/w.rb
I am willing to change this if someone has a better idea.
Optional:
-s # will set using spaces with the size of an indent being #, the default is a tab
-b will create a backup of the file that is filename+",bk."+#{Time.now} I suggest using this when you run it for the first time.
ruby simple_formatter.rb -f /home/sbecker/test_file.rb
The only output should be "Done!"
ruby simple_formatter.rb -s 2 -f /home/sbecker/test_file.rb
Same formatted file now with two spaces instead of tabs
ruby simple_formatter.rb -b -f /home/sbecker/test_file.rb
Creates formated file as test_file.rb and the original file test_file.bk.#{Time.now}
I will be updating the code as a find more things that break it. Right now its worked for 95% of the code i tested it on. The only thing i know it does not work with currently is one line do end statements.
To resolve some issues with other team members I have created a simple ruby formatter. I call it simple because it only formats one line at a time. There is not looking around at different lines, no knowledge of location in an object, or even scoping. I would like to create a rbchecker that would be like Pychecker, but this project is not that advanced. There is no parsing of ruby syntax, instead I am just looking for simple patterns. I am sure I missed a pattern or two but I have the option to create a back of the original file. I suggest turning on the back up for the first time you run this.
After I finished up a few test i figured i would look around to see if there were any other formatters. Turns out there are a few but, sine I am already done... Why not release it anyways?
Right now it is mostly comments, and no i did not run the formatter on it's self. I have been testing it on other projects of mine and koders.com ruby files.
Where to get it:
My svn http://svn.stephenbeckeriv.com/code/ruby_formatter/
How to use it:
Required:
-f file/location/e.rb
This can also be called with commas
-f file/location/e.rb,file/location/q.rb,file/location/w.rb
I am willing to change this if someone has a better idea.
Optional:
-s # will set using spaces with the size of an indent being #, the default is a tab
-b will create a backup of the file that is filename+",bk."+#{Time.now} I suggest using this when you run it for the first time.
ruby simple_formatter.rb -f /home/sbecker/test_file.rb
The only output should be "Done!"
ruby simple_formatter.rb -s 2 -f /home/sbecker/test_file.rb
Same formatted file now with two spaces instead of tabs
ruby simple_formatter.rb -b -f /home/sbecker/test_file.rb
Creates formated file as test_file.rb and the original file test_file.bk.#{Time.now}
I will be updating the code as a find more things that break it. Right now its worked for 95% of the code i tested it on. The only thing i know it does not work with currently is one line do end statements.
Ruby class mixin
Some times in ruby you want your mixin to be a class method. Search the google for ruby class mixin returned little help on the matter. Although, if you have never did a mixin before I suggest reading the pick axe book. This example should look familiar if you have read the book
Currently I can not think of a good example of when this would be used, but it can be done. I understand why this works, but I think there would be a better way to achieve this.
#!/usr/bin/env ruby module Debug def new_method() new() end end class Phonograph class << self include Debug end end class EightTrack class << self include Debug end end puts Phonograph.new_method puts EightTrack.new_method
Currently I can not think of a good example of when this would be used, but it can be done. I understand why this works, but I think there would be a better way to achieve this.
Ruby formatter 0-5-0
Good News Everybody! Ruby formatter has an raa site! But the real good news is Andrew Nutter-Upham submitted a patch that changes how the options are done!
I like this formatting a lot better. You can also do a --help (if you have RI installed) and it produces something (I do not have RI installed). I hope to stay active with this project and look forward to hearing from more people.
ruby [options] filelist
options:
-s # will change the indent to a space count of # per level
by default we space with 1 tab per level
-b # create a backup file
examples:
ruby simple_formatter.rb -s 3 -b /moo/cluck/cow.rb
runs with the indent of 3 spaces,creates a backup file, and formats moo/cluck/cow.rb
ruby simple_formatter.rb -s 31 /moo/cluck/cow.rb /beef/lo/mein.rb
formats both files with 31 spaces per level.
I like this formatting a lot better. You can also do a --help (if you have RI installed) and it produces something (I do not have RI installed). I hope to stay active with this project and look forward to hearing from more people.
Ruby formatter base
There are a few basic rules I have for the simple formatter.
1) Key words are followed by a space
So no if(true). I want if (true)
2) Comments will be aligned with the current depth
I tried not formatting the comments depth, but I did not like the results
3) Block comments and here docs(soon) are left unformatted
The begin and end lines will be aligned but the inside of the comment/doc will not be touched
Thats about it right now. A few problems problems I fixed was using words like definition, comments after statements, and the order in which I strip user data. The current code in svn fixes these problems.
I have a few debugging statements or a debug mode if you will. Just set DEBUG_ME to true and watch some logic unfold.
1) Key words are followed by a space
So no if(true). I want if (true)
2) Comments will be aligned with the current depth
I tried not formatting the comments depth, but I did not like the results
3) Block comments and here docs(soon) are left unformatted
The begin and end lines will be aligned but the inside of the comment/doc will not be touched
Thats about it right now. A few problems problems I fixed was using words like definition, comments after statements, and the order in which I strip user data. The current code in svn fixes these problems.
I have a few debugging statements or a debug mode if you will. Just set DEBUG_ME to true and watch some logic unfold.
Ruby formatter 0-6-0
I have been busy with the ruby formatter. I made the formatter a class. By class I mean I took the code from simple_formatter.rb and created some methods. I did this so people can create their own interfaces for the formatter.
Example:
I also added a test file that I will add code to if someone sends me a chunk of code that breaks the formatter
I also fixed a few thing:
Here docs are not formatted
while statements at the end of blocks work
checks for one-line rescue statements
better command line options
better debugging statments and use of profiler
I strip out all regex, strings and comments before checking ruby code
Next step? Find more code to break the formatter and improve performance.
Example:
format_me = Formatter.new ruby_code =<<-RUBY def moo p ouch end RUBY puts format_me.format_string(ruby_code) format_me.format_file("/x/y/z.rb")
I also added a test file that I will add code to if someone sends me a chunk of code that breaks the formatter
I also fixed a few thing:
Here docs are not formatted
while statements at the end of blocks work
checks for one-line rescue statements
better command line options
better debugging statments and use of profiler
I strip out all regex, strings and comments before checking ruby code
Next step? Find more code to break the formatter and improve performance.
Ruby formatter
I have been using profile to find places to improve my code. I was looking at the profiler when I was in debug mode but, in debug mode I call methods more then once. After turning the profiler on and the debug statements off I was able to locate methods that were called more then once per line of formatted code. I also changed some if statements around and added constant regxs.
I saved a log my changes and put them up on the svn
I saved a log my changes and put them up on the svn
Action mailer recieving attchments
So I posted before about receiving voice mail and having populate on the internet. Well step one is checking the mail. As it turns out my email for my server is stored in a local dir. This is really good news because it appears that action mailer does not check email. First I need to add a receive method to my mail model:
Done! But you say it does nothing? I know. I hate changing code in models and then restarting the server. For now my logic will stay in the controller:
Now that I have access to the mail object and I can save the attached file I can now store the necessary data for my website.
For a list of methods on the TMail object try this website http://railsmanual.com/class/TMail::Mail
TMail
def receive(email)
email
end
Done! But you say it does nothing? I know. I hate changing code in models and then restarting the server. For now my logic will stay in the controller:
def test #open the local file and read it in. x=Notifier.receive(File.read("/home/giggles/realybadsite/1184671181.V9I760a7f6.popeyemail.com")) #at this point x is a TMail::Mail (links below) # x.attachments will be an array that has a TMail::Attachment which inherits from StringIO # This holds the attachment ruby/gems/1.8/gems/actionmailer-1.3.2/lib/action_mailer/vendor/tmail/attachment.rb # it also has original_filename and content_type File.open(Dir.pwd+"/tmp/"+x.attachments.first.original_filename,"w+"){|file| file << x.attachments.first.gets } end
Now that I have access to the mail object and I can save the attached file I can now store the necessary data for my website.
For a list of methods on the TMail object try this website http://railsmanual.com/class/TMail::Mail
TMail
Bad tag cloud
I made a bad tag cloud. Click the tag link at the top.
Lazy site syntax
I use the wonderful gem call syntax to convert my ruby code to pretty html blocks. My biggest problem with this is that I wrote my own CRUD interface. I have no fancy helpers for formatting or links. I thought that ruby code formatting would be a good first start. As I thought about it some more why should I need to convert the ruby when I am creating the post. Why can't my code convert it when it displays it? Lets try it..
All I need to do is wrap my ruby code in a code tag
I Could have had the converter code not be its own method, but I plan on using cvert else where in code. I also want to be able to set type attributes in my code tags that can change the converter.
All I need to do is wrap my ruby code in a code tag
if data =~ /<code>.*<\/code>/m data.gsub!(/<code>.*<\/code>/m){|code| code.gsub!(/<[\/]*code>/,""); cvert(code.strip)} end def cvert(string) convertor = Syntax::Convertors::HTML.for_syntax "ruby" return convertor.convert( string ) end
I Could have had the converter code not be its own method, but I plan on using cvert else where in code. I also want to be able to set type attributes in my code tags that can change the converter.
Formatter do fix
Someone emailed me today about my formatter being broken. I was incorrectly looking for do statements. For example I would only catch do |x| and not just do. I was also sending the string to the one liner check. These have been fixed and committed.
Rails upload file test
I wrote a rails service that can take a file as the posted params or data in the raw body. Like the good developer I am I wanted to create unit test for this functionally. Unsure how I could do this I search the Google for terms including but not limited to upload test file post rails with no good results. I did find fixture_file_upload which I now question what it really does.
I should have known the solution from odd test mishaps I have had before.
When you set params in the testing frame work they are passed as is.
For example:
This will set param[:file] to a Tempfile like a normal mutil-part form post would. If you did not find this out already Rails will not always return you a Tempfile for posted files. If the file is small enough it will be a StringIO object. If you are reading the contents they will function the same real_data = params[:file].read . If you plan on resending the data you might want to check the kind_of? on the object that was passed in.
anyway good to know stuff.
I should have known the solution from odd test mishaps I have had before.
When you set params in the testing frame work they are passed as is.
For example:
x = Tempfile.new() x.write = temp_file_data x.rewind post :service_x,:file=>x,value=>"x"
This will set param[:file] to a Tempfile like a normal mutil-part form post would. If you did not find this out already Rails will not always return you a Tempfile for posted files. If the file is small enough it will be a StringIO object. If you are reading the contents they will function the same real_data = params[:file].read . If you plan on resending the data you might want to check the kind_of? on the object that was passed in.
anyway good to know stuff.
Programming challenges
My roommate has a nice book called Programming Challenges. Today I started to work on the problems in this book. It's a simple way to keep the logic muscles in shape. I will not be putting the questions in the ruby files just the answers. I have the book from 2003 and it does not talk about using ruby for these but hey its what I know best. The link to svn has a %20 in it I am thinking about changing it.
Also, my search for an algorithm book is over. I am going to take "Introduction to Algorithms" ISBN 0-262-03293-7 and make all the sudo code and example in ruby. I was searching for a good book and I think I finally found one.
Also, my search for an algorithm book is over. I am going to take "Introduction to Algorithms" ISBN 0-262-03293-7 and make all the sudo code and example in ruby. I was searching for a good book and I think I finally found one.
Rjs partials
I was asked if one can do partial rjs files by a coworker recently. Some of our RJS code is becoming large and difficult to maintained. I figured why should RJS be different then the other templates. So I started up my test rails application. This is very filled with 1/2 crazy ideas and 1/2 unsure syntax examples. So I added two calls rjs1 and rjs2. I created a link_to_remote[1] on rjs1 pointing to rjs2.
I did nothing in rjs2 but created an rjs2.rjs file in side of my views directory. My rsj2.rjs file looks like
_moo.rjs looks like
What I do not understand is why firebug[2] only shows me the first replace call in the response.
What it is close to what I expect to but not complete. This is what I think I should see:
I am happy because this can help clean out view logic from the controller RJS calls, but I am also concerned that the improper display in firebug[3] can have a negative impact on developing JavaScript. I am also thinking that this can be done in a different way.
[1] http://api.rubyonrails.org/classes/ActionView/Helpers/PrototypeHelper.html [2] http://www.getfirebug.com/ [3] Firebug display issue
page.replace_html "one","one" page<<"#{render(:partial=>"moo")}"
_moo.rjs looks like
I understand that render will return the JavaScript of the rjs rendering. I also understand that I need to use page<< to append the JavaScript to the call like I would use <%= %> in an RHTML file. Would appear to work, but it seems off.page.replace_html "two","two"
What I do not understand is why firebug[2] only shows me the first replace call in the response.
try {
Element.update("one", "one");
} catch (e) { alert('RJS error:\n\n' + e.toString()); alert('Element.update(\"one\", \"one\");'); throw e }
What it is close to what I expect to but not complete. This is what I think I should see:
try {
Element.update("one", "one");
try {
Element.update("two", "two");
} catch (e) { alert('RJS error:\n\n' + e.toString()); alert('Element.update(\"two\", \"two\");'); throw e }
} catch (e) { alert('RJS error:\n\n' + e.toString()); alert('Element.update(\"one\", \"one\");'); throw e }
Note the nested try catch is only there because of the rjs debug setting. So I changed debug to false and restarted the server. I now see this in firebug:
Element.update("one", "one");
Element.update("two", "two");
I am happy because this can help clean out view logic from the controller RJS calls, but I am also concerned that the improper display in firebug[3] can have a negative impact on developing JavaScript. I am also thinking that this can be done in a different way.
[1] http://api.rubyonrails.org/classes/ActionView/Helpers/PrototypeHelper.html [2] http://www.getfirebug.com/ [3] Firebug display issue
Ruby documentation
I am looking for documentation on this functionality in ruby.
Clearly it returns the numeric value of 'g'. Why would I use this feature I can not find documentation about? Speed..104 == ?g #=> true
I am looking around my small set of books and the online documentation and have not found anything. I must admit I am unsure how to search for this.n = 5000000 => 5000000 Benchmark.bm do |x| x.report { 0.upto(n) do 104 == ?g end} x.report { 0.upto(n) do "g" == "g" end } x.report { 0.upto(n) do "g".eql?("g") end} end # user system total real # 1.270000 0.000000 1.270000 ( 1.330858) # 4.220000 0.010000 4.230000 ( 4.497720) # 4.190000 0.070000 4.260000 ( 5.206511)
Ruby documentation
I found the answer to my ruby code question. I found the documentation for assignment using *.
*x = one,two,three #=> x=[one,two,three] one,two,three = *x #=> this takes the x array and splits it out.
Rails rendering
No doubt any one looking at rails for speed will find that rendering sucks up a fair bit of time. I have two ideas to change the rendering system. The first is to remove the benchmarking code for production. I have no proof that this will make things faster. I am just sick of seeing it in my profiling log. The second idea I have is allowing the user of rails to change out rendering engines. For example using markaby instead of ERB or libxml(c lib) instead of XMLbuilder. I think using different libs could yield a faster render. If not with a c lib replacement then with another lib yet to be written.
Hotp
In my final days of work I have created a library for the HMAC-Based One-Time Password(HOTP) Algorithm. HOTP was created in hopes that a freely available encryption standard would generate more two-factor authentication devices. Looking around on the net you will find that there are a handful of devices that support HOTP. If you would like to read more about HOTP and its inner workings check out RFC 4226 (http://www.ietf.org/rfc/rfc4226.txt)
As for ruby and HOTP it is done. I hope to have a project up on RAA and rubyforge soon. A coworker helped me fill out the class, create a gem and correct a few implementation errors.
Example 1:
I think the RFC has you generate hotp numbers until you match the one that was passed in or you hit a max number of tries. This number would be inputed along with a personal pin and a unique id that you can related to the secret text for that card.
Example 2:
You get the point. I will post the links to the code when they are up.
As for ruby and HOTP it is done. I hope to have a project up on RAA and rubyforge soon. A coworker helped me fill out the class, create a gem and correct a few implementation errors.
Example 1:
#example numbers from the RFC require "HOTP" h = HOTP.new() h.secret = "12345678901234567890" h.count = "0" #can be a string or number. h.digits = 6 h.update #=> "755224" h.hotp #=> "755224"
I think the RFC has you generate hotp numbers until you match the one that was passed in or you hit a max number of tries. This number would be inputed along with a personal pin and a unique id that you can related to the secret text for that card.
Example 2:
#example numbers from the RFC require "HOTP" search_digits = ARGS[0] (0..10000).each{|counter| if HOTP::hotp("12345678901234567890",counter) == search_digits puts "you pressed the button #{counter} times!" break end }
You get the point. I will post the links to the code when they are up.
Hotp project
The HOTP code is released. You can gem install it because it is hosted on rubyforge. I also made an announcement on ruby talk. The Hotp class implements RFC 4226 in Ruby. (http://www.ietf.org/rfc/rfc4226.txt) HOTP is an HMAC-SHA1 based algorithm for one time passwords. The README has examples on how to use the library. We used the perl implantation and the RFC to write our test.
http://rubyforge.org/projects/hotp/ and http://raa.ruby-lang.org/project/hotp
http://rubyforge.org/projects/hotp/ and http://raa.ruby-lang.org/project/hotp
Login generator
I have posted about my ruby on rails generator that generates generators before. When I created this I did not know the generator code that well. Sadly this is still the case. I used generators for my one task saving me from created a nearly similar files over and over again. But Becker could you have not metaprogramed it? No each model had a common idea and interface but in the end the code was very different. I could have reduced the code and created a mixin for most of the methods, but it's a little late for that. Also I do not think I want to create a mixin for code that should be changing a lot.
Long story shorter I created some generators. I found my self creating another rails app for my websites and I am writing the login code over and over again. I thought why not generate? Then I thought why has someone not done this? But they have and, each of the generators work. They all have good attributes and bad ones but, none of them are 100% what I wanted. So I am going to make my own login generator. I will post it in my svn when its ok. I also realize the literature on generators is lacking. I also feel that the amount of generators that have been created is very low. It might be the most under used feature in RoR.
Long story shorter I created some generators. I found my self creating another rails app for my websites and I am writing the login code over and over again. I thought why not generate? Then I thought why has someone not done this? But they have and, each of the generators work. They all have good attributes and bad ones but, none of them are 100% what I wanted. So I am going to make my own login generator. I will post it in my svn when its ok. I also realize the literature on generators is lacking. I also feel that the amount of generators that have been created is very low. It might be the most under used feature in RoR.
Locomotive woes
After week one at Trusera I finally have my Mac configured to not use Locomotive. Locomotive is a wonderful little package of very common libraries for Rails on the Mac. It creates a sandbox of the libraries in some crazy tmp directory. I am confused on how they install Rmagick or ImageMagick with out really installing the libraries that are needed. I am concerned that the code I was using would not work out side the context of Locomotive even with the proper libraries and gems installed.
A fun problem to track down was during the booting of Rails via webrick in the Komodo IDE debugger I ran in to an error "constant RedHillConsulting::ForeignKeyMigration should be declared". I am not sure it was related to running webrick in Komodo, but I fixed this by requiring the RedHill libraries inside of the init.rb file for ForeignKeyMigration.
Other then that the other libraries installed very easily on the OSX 10.5 and I can use komodo's debugger.
A fun problem to track down was during the booting of Rails via webrick in the Komodo IDE debugger I ran in to an error "constant RedHillConsulting::ForeignKeyMigration should be declared". I am not sure it was related to running webrick in Komodo, but I fixed this by requiring the RedHill libraries inside of the init.rb file for ForeignKeyMigration.
Other then that the other libraries installed very easily on the OSX 10.5 and I can use komodo's debugger.
Firewatir
I never like to repeat the same tasks over and over again. I am sure this is why I am a programmer. With that said, I love to automate or streamline my daily/monthly processes. I have used Capistrano to create packages , copy files, and restart services. I have also created Selenium scripts to test website functionally.
My first week at work was mostly QA time and getting to know the code. One of tasks was to look in to automating test plans. I spent about a day trying to get Selenium working on my mac with no luck at all. So I started looking in to Watir(water). It looks like it is a very nice tool but, it is mainly a Windows based application. There is a Firefox only version called Firewatir(firewater) that works on window and OSX. I am unsure how I feel about the library api's but it is functional. Right now I find a lot of Watir docs written for the IE version, but I do not think the Firefox version is 100% yet.
It will be a slow learning process. Also I am learning a great deal about designing a website with automated testing in mind.
My first week at work was mostly QA time and getting to know the code. One of tasks was to look in to automating test plans. I spent about a day trying to get Selenium working on my mac with no luck at all. So I started looking in to Watir(water). It looks like it is a very nice tool but, it is mainly a Windows based application. There is a Firefox only version called Firewatir(firewater) that works on window and OSX. I am unsure how I feel about the library api's but it is functional. Right now I find a lot of Watir docs written for the IE version, but I do not think the Firefox version is 100% yet.
It will be a slow learning process. Also I am learning a great deal about designing a website with automated testing in mind.
Virtual server
I am using dreamhost's virtual servers. I am currently limited to 150mb of ram and a 150mhz cpu. This should be more then enough for my little web servers. I found after 10 days or so all my ram was being used. I think it is because of zombie dispatchers but I can never trouble shoot it to find out. I just added some tools that will allow me to see what is going on.
Oddly enough, I can not get top to run from rails. The top command will normally wait for input and keep updating frames. If you use -b -n1 it runs in "batch" mode and will cut out after one frame. This worked fine from irb and I was able to save the data to a string. In rails this fails, but I can ls, and ps. I normally only use ps to find a pid I want to kill. So I only know the -elf options and I grep for my data. After reading the man page for ps I found out I can out put the % of memory a pid is using. This was perfect. I will be able to tell exactly what the memory hog is.
Oddly enough, I can not get top to run from rails. The top command will normally wait for input and keep updating frames. If you use -b -n1 it runs in "batch" mode and will cut out after one frame. This worked fine from irb and I was able to save the data to a string. In rails this fails, but I can ls, and ps. I normally only use ps to find a pid I want to kill. So I only know the -elf options and I grep for my data. After reading the man page for ps I found out I can out put the % of memory a pid is using. This was perfect. I will be able to tell exactly what the memory hog is.
Woot and hpricot
I want to learn more about one of my favorite sites, Woot. It was the idea of my stat friend Drew, that he might be able to predict a woot off if he had the data about woot items. Easy enough to parse a website. After looking around woot for a few minutes I found that the blog holds all the data I needed. Lucky enough the data goes back to may of 2006.
Lets start..
Lets start..
Each blog page holds about 20 Woot items. It turns out that there are 58 pages of items. The Woot off days are days with more then one item.#!/usr/bin/env ruby require "rubygems" require "hpricot" require "open-uri" #my base blog url uri_base = "http://www.woot.com/Forums/Default.aspx?p=" #When parsing the woot pages I got an error and after a google search I found I needed to increase the hpricot buffer Hpricot.buffer_size = 262144
After this I create a CSV file with data. This allows me to re-parse the data later for anything.master_list = [] (0..58).each{|page| #open the page doc = Hpricot(open(uri_base+page.to_s)) #get all item rows row =(doc/'tr.itemRow') data =[] #create the date and title entries data = row.map{|row| ["#{(row/'div.saleMonth').text} #{(row/'div.saleDay').text} #{(row/'div.saleYear').text}","#{(row/'div.saleTitle/a').text}"] } #add the values of the data array not the array master_list.push(*data) #dont kill the server sleep(5) }
My code and data files are in my SVN.File.open("woot.data","w+"){|file| file<<"Date,Item\n" master_list.each{|woot_item| file.puts woot_item[DATE]+","+woot_item[ITEM].gsub(",","\,")+"\n" } }
Rails and soap4r
Recently at work I was tasked with making rails work with soap4r. There is a ticket on the soap4r trac[1] and in the Rails trac[2]. There are a wide range of solutions. I personally try to avoid changing core code if I can. With that in mind I have created two solutions.
In both solutions you need to not have soap4r install in the default gem folder. So gem install it in the Rails layout.
Solution 1:
Set the GEM_PATH in the rails project:
Setting gem_path after require rubygems is a great way to keep soap4r in one project. The gem library is to use once you find the documentation. For some reason it is not listed on the ruby-doc website but you should have a local version of the docs. In your environment.rb file add
Solution 2:
Use rubygem classes:
rubygems is just another ruby class to load other ruby classes. Its one of the reasons we are in this mess to begin with. The good news is I have searched for example code and found nothing useful
[1] http://dev.ctor.org/soap4r/ticket/433 [2] http://dev.rubyonrails.org/ticket/10001
In both solutions you need to not have soap4r install in the default gem folder. So gem install it in the Rails layout.
#since having soap4r installed will break all rail projects remove it. sudo gem uninstall soap4r sudo gem install --install-dir ~/rails_dir/vendor/gems soap4rYou can update the gem and remove it if needed. I see two negatives to this solution. One is that when I run IRB I do not have soap4r. This is solved by having it also installed in the normal gem dir on my dev box. Two there is going to be extra bandwidth with coping your rails code around. This can also be solved by installing it on your production boxes. If you have more then one prod box that could be a pain to remember to do.
Solution 1:
Set the GEM_PATH in the rails project:
Setting gem_path after require rubygems is a great way to keep soap4r in one project. The gem library is to use once you find the documentation. For some reason it is not listed on the ruby-doc website but you should have a local version of the docs. In your environment.rb file add
or the location you installed the gems to. Then gem install your little hard way.ENV['GEM_PATH'] = "#{File.dirname(__FILE__)}/../../../../gems" require "rubygems" Gem.clear_paths
Solution 2:
Use rubygem classes:
rubygems is just another ruby class to load other ruby classes. Its one of the reasons we are in this mess to begin with. The good news is I have searched for example code and found nothing useful
gems = Gem::SourceIndex.from_gems_in("~/local_gems")
spec = gems.search('soap4r').last
# straight from the gem requiring code base
# bin directory must come before library directories
spec.require_paths.unshift spec.bindir if spec.bindir
soap_path = ""
require_paths = spec.require_paths.map do |path|
tmp = File.join spec.full_gem_path, path
soap_path = tmp if tmp.include?("/lib")
tmp
end
# straight from the gem requiring code base
sitelibdir = Config::CONFIG['sitelibdir']
# gem directories must come after -I and ENV['RUBYLIB']
$LOAD_PATH.insert($:.index(sitelibdir), *require_paths)
#for some reason I needed to require path/soap/soap I am sure I can fix that
require File.join(path,"soap","soap")
[1] http://dev.ctor.org/soap4r/ticket/433 [2] http://dev.rubyonrails.org/ticket/10001
Sort vs sort!
I remembered reading a comment Matz made once about the ! methods. It went something like "! methods are destructive and should only be used when performance is needed". I found a benchmark example that show sort vs sort! speeds are depended on the garbage collection[1]. Today I was reading a chunk of ruby-prof code and found the comment.
I rewrote the code with a sort! in my head but wonder would it really have any affects on the time? A little context for test. I just used ruby-prof in an around filter in the application.rb file for a rails project. The html file that ruby-prof produces is not small and includes parts of the rails framework, mysql library, and core ruby code.
I just submitted a patch to the ruby prof ruby forge group[2]. It is not a big improvement over all. Using my numbers above the current GraphHtmlPrinter use 0.013572 just for sorting. It should now only take 0.004242. Saving the world most a second....
[1] http://www.ruby-doc.org/core/classes/Benchmark.html
[2] http://rubyforge.org/tracker/index.php?func=detail&aid=16364&group_id=1814&atid=7062
#from RubyProf::GraphHtmlPrinter def calculate_thread_times # Cache thread times since this is an expensive # operation with the required sorting @result.threads.each do |thread_id, methods| top = methods.sort.last thread_time = 0.01 thread_time = top.total_time if top.total_time > 0 @thread_times[thread_id] = thread_time end end
I rewrote the code with a sort! in my head but wonder would it really have any affects on the time? A little context for test. I just used ruby-prof in an around filter in the application.rb file for a rails project. The html file that ruby-prof produces is not small and includes parts of the rails framework, mysql library, and core ruby code.
Clearly not enough improvement to stop caching thread times but, now the methods are in order. We could stop caching and access the thread time likerequire "benchmark" def calculate_thread_times!(result) thread_times={} result.threads.each do |thread_id, methods| methods.sort! thread_time = methods.last.total_time > 0 ? methods.last.total_time : 0.01 thread_times[thread_id] = thread_time end end def calculate_thread_times(result) # Cache thread times since this is an expensive # operation with the required sorting thread_times ={} result.threads.each do |thread_id, methods| top = methods.sort.last thread_time = 0.01 thread_time = top.total_time if top.total_time > 0 thread_times[thread_id] = thread_time end end Benchmark.bm(12) do |test| test.report("no_bang:") do lresult = result calculate_thread_times(lresult) end test.report("bang:") do lresult = result calculate_thread_times!(lresult) end end # user system total real #no_bang: 0.010000 0.000000 0.010000 ( 0.006786) #bang: 0.000000 0.000000 0.000000 ( 0.004242)
Since the calculate_thread_times is called when you create a new RubyProf::GraphHtmlPrinter you should never need to call sort again. For example in the ERB template. I did find that using Benchmark.bmbm cause the real values to be swapped just like in the rdoc example[1].@result.threads[thread_id].last #I think
I just submitted a patch to the ruby prof ruby forge group[2]. It is not a big improvement over all. Using my numbers above the current GraphHtmlPrinter use 0.013572 just for sorting. It should now only take 0.004242. Saving the world most a second....
[1] http://www.ruby-doc.org/core/classes/Benchmark.html
[2] http://rubyforge.org/tracker/index.php?func=detail&aid=16364&group_id=1814&atid=7062
Large numbers of n
I found my self need to find the greatest number in an array of random numbers as fast as I can. Which sounds like a comp sci 101 problem visit every node and keep track of the highest number. I came up with a few different ways to do this with ruby's arrays and set up benchmarking.
Some might say its not fair. That I changed the out come by measuring it, but I had to do it. Thats right sorting a random array to find the largest value of that array is the quickest way I can find. I have also tried this with 60k and 600k arrays with the same relative speeds. Since Ruby 1.8 it has used the Schwartzian transform[1] for its sort method. Which is clearly faster then each but why is each_index twice as slow as each? I thought the size indexing test was slower because of the range object creation but I tried it with a precompiled range and it still was 8 seconds.
[1] http://en.wikipedia.org/wiki/Schwartzian_transform
And the winner is Schwartzian transform with max as a close second!n = 1000 marray = Array.new(6000) range = (0...6000) range.each{|i| marray[i]=rand(4000000)} require "benchmark" Benchmark.bmbm(1) do |test| test.report("Sort!:") do n.times{ array =Array.new(marray) array.sort! array.last } end test.report("max:") do n.times{ array =Array.new(marray) array.max } end test.report("Sort:") do n.times{ array =Array.new(marray) array.sort.last } end test.report("range index:") do n.times{ max=0 array =Array.new(marray) range.each{|x| max=array[x] if array[x]>max} } end test.report("index:") do n.times{ max=0 array =Array.new(marray) array.each_index{|x| max=array[x] if array[x]>max} } end test.report("size indexing:") do n.times{ max=0 array =Array.new(marray) (0...array.size).each{|x| max=array[x] if array[x]>max} } end test.report("each:") do n.times{ max=0 array =Array.new(marray) array.each{|x| max=x if x>max} } end test.report("uniq each:") do n.times{ max=0 array =Array.new(marray) array.uniq! array.each{|x| max=x if x>max} } end test.report("pop:") do n.times{ max=0 array =Array.new(marray) while x=array.pop max = x if x>max end } end end
#Rehearsal -------------------------------------------------- #Sort!: 1.380000 0.050000 1.430000 ( 1.462959) #max: 1.920000 0.010000 1.930000 ( 1.932106) #Sort: 1.380000 0.030000 1.410000 ( 1.409176) #range index: 8.060000 0.030000 8.090000 ( 8.121491) #index: 8.100000 0.070000 8.170000 ( 8.200113) #size indexing: 8.120000 0.060000 8.180000 ( 8.194519) #each: 4.840000 0.020000 4.860000 ( 4.884708) #uniq each: 7.140000 0.060000 7.200000 ( 7.203439) #pop: 5.590000 0.020000 5.610000 ( 5.611432) #---------------------------------------- total: 46.880000sec # user system total real #Sort!: 1.380000 0.040000 1.420000 ( 1.415057) #max: 1.920000 0.020000 1.940000 ( 1.927911) #Sort: 1.380000 0.040000 1.420000 ( 1.418863) #range index: 8.050000 0.020000 8.070000 ( 8.060964) #index: 8.070000 0.010000 8.080000 ( 8.079052) #size indexing: 8.070000 0.030000 8.100000 ( 8.085902) #each: 4.820000 0.010000 4.830000 ( 4.825778) #uniq each: 7.160000 0.070000 7.230000 ( 7.271699) #pop: 5.600000 0.030000 5.630000 ( 5.662141)
Some might say its not fair. That I changed the out come by measuring it, but I had to do it. Thats right sorting a random array to find the largest value of that array is the quickest way I can find. I have also tried this with 60k and 600k arrays with the same relative speeds. Since Ruby 1.8 it has used the Schwartzian transform[1] for its sort method. Which is clearly faster then each but why is each_index twice as slow as each? I thought the size indexing test was slower because of the range object creation but I tried it with a precompiled range and it still was 8 seconds.
[1] http://en.wikipedia.org/wiki/Schwartzian_transform
Iteration woes
My world has just been turned upside down. I need to start benchmarking some real data. I am using large arrays of random. Anyway, after trying to find the max number in array I found using the each method was slower then sorting. I then wondered if different types of iteration would be faster. So after some wings for breakfast at the Seattle airport I created another benchmark.
And the results?
If I left out any ways to iterate please let me know.
n = 10000 start_array = Array.new(6000) (0...6000).each{|i| start_array[i]=rand(4000000)} require "benchmark" Benchmark.bm(1) do |test| test.report("each:") do n.times{ start_array.each{|x| x } } end test.report("for:") do n.times{ for x in start_array x end } end test.report("each_index:") do n.times{ start_array.each_index{|i| start_array[i] } } end test.report("range each:") do n.times{ (0...start_array.size).each{|i| start_array[i] } } end test.report("each_with_index:") do n.times{ start_array.each_with_index{|x,i| x } } end end
And the results?
# user system total real #each: 14.360000 0.070000 14.430000 ( 14.516698) #for: 11.880000 0.040000 11.920000 ( 11.919000) #each_index: 44.280000 0.160000 44.440000 ( 44.451407) #range each: 44.760000 0.190000 44.950000 ( 45.039539) #each_with_index: 46.560000 0.320000 46.880000 ( 47.269472)
If I left out any ways to iterate please let me know.
Different includes
So there are many different ways to tell if an array includes an object. The results will change depending on where in the array the object is and if the object is in the array or not. A I once again used a random array to benchmark a few ways to do this.
I ran the test again but this time I inserted 5 at index 200. No shock they all cut out when they find the object they are looking for.
The results?n = 10000 start_array = Array.new(6000) (0...6000).each{|i| start_array[i]=rand(4000000)} require "benchmark" Benchmark.bm(1) do |test| test.report("include?:") do n.times{ start_array.include?(5) } end test.report("for loop:") do n.times{ found = false for x in start_array if x == 5 found = true break end end } end test.report("any:") do n.times{ start_array.any?{|x| x==5} } end test.report("find:") do n.times{ true if start_array.find{|x| x==5} } end test.report("select:") do n.times{ true if start_array.select{|x| x==5} } end end
# user system total real #select: 52.730000 0.400000 53.130000 ( 53.264748) #include?: 12.180000 0.020000 12.200000 ( 12.222207) #for loop: 44.630000 0.100000 44.730000 ( 44.863327) #any: 52.890000 0.450000 53.340000 ( 53.563788) #find: 43.590000 0.350000 43.940000 ( 43.999914)
I ran the test again but this time I inserted 5 at index 200. No shock they all cut out when they find the object they are looking for.
# user system total real #include?: 0.410000 0.010000 0.420000 ( 0.449498) #for loop: 1.500000 0.010000 1.510000 ( 1.545805) #any: 1.780000 0.000000 1.780000 ( 1.777178) #find: 1.770000 0.000000 1.770000 ( 1.773765) #select: 43.750000 0.400000 44.150000 ( 44.936202)
New imdb idea
So I was reading some comments from dzone[1] about my imdb "api". I do not like that the imdb site can change and I am SOL until I fix the code. For what I use my imdb class for it is not an issue but for some other people it most likely will be an issue. I have started work on creating migration scripts and data loaders for the downloaded version of IMDB[2]. I hope this project does not join my my dead projects list[3].
Happy new year...
[1] http://dzone.com/links/ruby_imdb_api.html
[2] http://www.imdb.com/interfaces
[3] http://stephenbeckeriv.com/dead_projects
Happy new year...
[1] http://dzone.com/links/ruby_imdb_api.html
[2] http://www.imdb.com/interfaces
[3] http://stephenbeckeriv.com/dead_projects
Futurama game
At my first seattle.rb meeting some people showed off the demos of Gosu[1], the ruby game enginey thingy. Anyway, with my passion for futurama being what it is I started on my first Futurama game! Also note this is the first time I am including a pic in a post.
With 147 lines of ruby I can fly the Planet express ship around the screen and fire lazers! Thats right lasers with a z. Anyway I plan on having some space pirate or space bandito ships flying around that you have to shoot. So far I just changed a demo graphics and added in bullets. I have an epic plan for my game. It will start like asteroids then change in to a Mario like side scroller then turn in to a defender like game. After that another Mario like level that ends at a pac-man like level that ends with space invaders.. If you know your Futurama its like the tales of interest show but a real game. I hope to have an svn repo soon. [1] http://code.google.com/p/gosu/
With 147 lines of ruby I can fly the Planet express ship around the screen and fire lazers! Thats right lasers with a z. Anyway I plan on having some space pirate or space bandito ships flying around that you have to shoot. So far I just changed a demo graphics and added in bullets. I have an epic plan for my game. It will start like asteroids then change in to a Mario like side scroller then turn in to a defender like game. After that another Mario like level that ends at a pac-man like level that ends with space invaders.. If you know your Futurama its like the tales of interest show but a real game. I hope to have an svn repo soon. [1] http://code.google.com/p/gosu/
Profanity filter of tomorrow
Announcing the Profanity Filter Of Tomorrow(PFOT) pronounce P-foot.
This is a web service call that will accept a posted text field and will return the text with the curses replaced or marked. Curses can include but not limited to obscenity, indecency and profanity. The service will be provided for one or many languages. Other services like spelling and grammar checking can also be added to the webservice call.
The service will accept and return different formatted responses including but not limited to Hyper Text Markup Language (HTML), JavaScript Object Notation (JSON), and Extensible Markup Language (XML).
Curse word dictionaries will be updated periodically by an aggregation of different sources including but not limited to humans and internet sources.
Why?
Although it is easy to program a simple white list to filter out profanity. There are a few problems with coding a white list. The first problem is the maintaining the list of words. Kids keep creating new ways to express those same old works. New slang, alternate spellings and fun elite(l33t) speak. A second problem is customization. I might want to filter some fields more or less strict or I might want to replace the bad words will fun silly words. I might want to replace the curses with curses from a different language. A third is foreign languages. Sure start filtering English and everyone will start cursing in French.
Stay tuned...
This is a web service call that will accept a posted text field and will return the text with the curses replaced or marked. Curses can include but not limited to obscenity, indecency and profanity. The service will be provided for one or many languages. Other services like spelling and grammar checking can also be added to the webservice call.
The service will accept and return different formatted responses including but not limited to Hyper Text Markup Language (HTML), JavaScript Object Notation (JSON), and Extensible Markup Language (XML).
Curse word dictionaries will be updated periodically by an aggregation of different sources including but not limited to humans and internet sources.
Why?
Although it is easy to program a simple white list to filter out profanity. There are a few problems with coding a white list. The first problem is the maintaining the list of words. Kids keep creating new ways to express those same old works. New slang, alternate spellings and fun elite(l33t) speak. A second problem is customization. I might want to filter some fields more or less strict or I might want to replace the bad words will fun silly words. I might want to replace the curses with curses from a different language. A third is foreign languages. Sure start filtering English and everyone will start cursing in French.
Stay tuned...
Cs forgotten
My memory was never the best. I have never worked on improving my memory because I can figure things out. If I do something enough it becomes second nature. The point is someone I work with was talking about Ternary search trees[1] and I had forgotten what they were. He started to describe what they are used for and how they basically work. I made the comment "it would be an interesting data structure in Ruby" which lead to the comment "Perl implements it with a hash of hashes." After the conversation the name of the tree left my mind and all I had was the perl comment and basic functionality. So I went to Cafe Presse[2] and start writing on a napkin while eating my Steak Frites. (I love those steak frites)
Basic functionally was defined as an interface that told me what words were in a dictionarythat was loaded, this is line for line what I wrote on the napkin
There was also a drawling of benders head and some flushed out hashes. It turns out this works well enough. Add in some load commands and different accessors and it should work well enough to get anyone started. The last bit of code that I added was an auto-complete part. I wrote this the next day at cafe vita over a double short americano(minus the comments)
Basic functionally was defined as an interface that told me what words were in a dictionarythat was loaded, this is line for line what I wrote on the napkin
@@top_hash = {} def create_hash(word) current_hash = @@top_hash word.strip! word.downcase! word.split(//).each{|letter| current_hash[letter] ||= {} current_hash = current_hash[letter] } current_hash["END"] = true end def is_word?(word) real_word = false word.downcase! word.strip! current_hash = @@top_hash (0...word.size).each{|i| current_hash = current_hash[word[i].chr] break unless current_hash real_word = true if ((i == word.size-1) && current_hash["END"]) } real_word end word = "gennosuke" create_hash(word) create_hash("ninja") create_hash("nine") pp @@top_hash pp is_word?(word) pp is_word?("cow")
There was also a drawling of benders head and some flushed out hashes. It turns out this works well enough. Add in some load commands and different accessors and it should work well enough to get anyone started. The last bit of code that I added was an auto-complete part. I wrote this the next day at cafe vita over a double short americano(minus the comments)
#get me to the end of the base then look for tails to add #there is improvements here if you can have compounded words def find_other_words(word_base) word_base.downcase! word_base.strip! current_hash = @@top_hash (0...word_base.size).each{|i| current_hash = current_hash[