Importing data with node.js into DB

Frustration

Node’s greatest strength is often the source of frustration for a node programmer. Most recently I was attempting to write a simple node.js script that took a csv file, parsed it and inserted it into a postgresql db. This is simple in ruby, python and many other scripting related languages but I soon hit a roadblock on node.js. The following script was my first attempt at importing the data.

Read More

Scheduled Events in Mysql

Justification

There are many options for scheduling events. Cron in Linux, scheduler in windows, every queueing systems has a built in schedule and then there are homegrown while loops. So why should you consider Mysql event.

  • Mysql is often already in your application
  • Its OS agnostic. It works the same whether its linux or windows
  • If the scheduled tasks is a db maintenance event it makes sense to store it on the db.
Read More

nodejs-as-a-windows-service

To install node.js as a windows service you need three items.

  • NSSM http://nssm.cc/download
  • node.js for windows . http://nodejs.org/download
  • Your node.js application.
Read More

Casing Up the Raspberry Pi, Part I

The Problem

Raspberry Pi is a great little computer but easily the biggest weakness of the board is its power and/or usb ports. Imagine getting your new raspbery pi, unboxing it, plugging it in with the usb charge you stole from your kid’s kindle, plugin your hdmi, keyboard and mouse and ….. nothing. Turns out a keyboard and a mouse can overwhelm your PI, and forget about that wifi adapter or the usb stick with all your rap mp3s. Youre just out of luck.

Read More

Setup Script Login

A recent trend is to build small headless computer systems based on linux. This has come from the ability to run linux on very small computers (in size and power) for very utilitarian tasks (routers, NAS, data collection nodes). However these use cases make it inconvient to connect a keyboard, mouse and monitor to the linux machine. Often its because the device may not have a video out. Or the device’s use case does not require a monitor/keyboard to be connected in its normal operations. So when maintenance and/or administrative processes are necessary you need to provide some sort of interface/UI. You can invest hours into developing a cool Web UI that a administrator uses once every 6 months but sometimes a menu driven terminal app could do the job.

Read More

Sharding a MultiTenant SaaS app

You’ve finally got some traction with your SaaS project. Lots of large customers and things are moving along rather well. A few of your customers complain that the system is not as fast as it use to be but that’s normal. You up the hardware, add a few indexes and things seem steady. All of a sudden your email/support staff is overwhelmed by customers complaining about 404 errors. You look and see that your web sessions are timing out and queries are starting to drag. Postgres Autovacuum is starting at the worse possible times and you are spending a lot of time babysitting your app instead of counting your money. You thought you had more time but now its obvious, its time to shard the database.

Read More

Redesign of Wi-spy Web interface.

So originally I wanted to build a web inteface for the wi-spy spectrum analyzer. I proved the concept with the first version. You can watch the short video yourself that goes over the design. The biggest problem with the application was that it required a lot of infrastructure. It required a special verion of spec_Tool to collect the data, node.js to supply the web server and a redis daemon to act inbetween. When I started adding code to store off historical data I realized I’d need another technology such as tokyo cabinet and the like. This proved even more difficult because I need a locking mechanism between the reader and the writer.

Read More

Why multi-field indexes in postgresql still matter.

Why wouldn’t it.

As of 8.1 postgresql includes the concept of a bitmap heap. A bitmap heap as implemented in postgresql is an array/matrix of bits stored in ram with each cell representing a record in the table being queried. Using the conditions included in the query the cell for each record is activated or deactivated. to show that these records should be included in the result of the query’s search. At the end of the query the bitmap heap is what determines wich records will be shown. This bitmap can be built by scanning different indexes based on the query criteria Compare this to how simpler databases like mysql work. In mysql only a single index is used for each table in each query.

Read More

Rocking ltree on postgresql

Postgresql 9.1 Now ships with a module called ltree. This is a cool postgresql that allows you to model hierarchies in you database and leverages the GIST indexing mechanism built into postgres. Assume you are modeling several organizations:

CREATE TABLE orgs (path ltree);
INSERT INTO orgs VALUES ('Top');
INSERT INTO orgs VALUES ('Top.OrgA');
INSERT INTO orgs VALUES ('Top.OrgA.Division1');
INSERT INTO orgs VALUES ('Top.OrgA.Division1.storeZ');
INSERT INTO orgs VALUES ('Top.OrgA.Division1.storeY');
INSERT INTO orgs VALUES ('Top.OrgA.Division2');
INSERT INTO orgs VALUES ('Top.OrgA.Division2.storeV');
INSERT INTO orgs VALUES ('Top.OrgA.Division2.storeW')
CREATE INDEX path_gist_idx ON test USING gist(path);
CREATE INDEX path_idx ON test USING btree(path); Now you can do some interesting queries such as Give me all the rstores in Division1
select * from orgs where path ~ "Top.OrgA.Division1.*" or give me all the stores (where the path is at least 4 items long)
select * from orgs where path ~ Top.{4}

This is another neat feature that Postgresql gives you but mysql doesn’t

Read More

adjusting innodb_log_file_size in mysql to get around

was getting "ERROR 1033 (HY000): Incorrect information in file: './realworx_production/analyzers.frm'" whenever I tried to adjust my mysql innodb parameters and run a query. To adjust this you have do delete or move the ib_logfile0 and ib_logfile1 file in your datadir. Then start the database.
Read More

Accessing a non standard database in rails

Normally in rails you would configure a production, test and development database in the database.yml file. If some of your rails data comes from a different database you can configure it in database.yml as well. For example I wanted two rails applications to synchronize their databases occassionally. To do this I created a remote db in one of the rails application that looks like this in database.yml:

Read More

Is apple trying to do an endrun around the desktop

A friend told me suggested apple was making a cradle that allowed you to connect your iphone to a monitor , keyboard and mouse. This is an interesting idea. Could apple try to make a dent into the desktop market by replacing the desktop. After all windows XP’s minimum requirements includes a 233 MHz processor. The iphone runs 600 MHz and Im sure the iphone’s OS is not nearly as bloted as XP. What is the real difference between an iphone and a Mac.

Read More

writeMultibyte for Flex on Linux

writeMultiByte is not working for linux (though it is fine in Windows) in Flashplayer 10.  It failed for me in the ByteArray class and I’ve seen other postings that the issue exist for the binarySocket class as well. At least for the us-ascii encoding.

A For loop around a writeByte is thework around

Read More

Radio Buttons act more like Combobox

Normally what I would do with radio buttons is to look for a click event and set variables accordingly.  However there is another way to use radio button if you are operating more in a ‘filling out a form’ approach.

Read More

Verify ItemEditors input in a datagrid.

Working a lot with ItemEditors in DataGrids.

One of the issues I’ve been dealing with is adding validation to the item editor text entry. I’ve been attaching my validations to the item editor objects themselves however  the Datagrid object throws out ItemEditEnd events. You have to check the event’s DataGridReason to see whether it is a cancel or not and then you have the opportunity to validate the data.  If the data is bad then do a event.PreventDefault to interrupt the data acceptance and set the target’s errorString to the error message.

More details can be found here.

http://www.adobe.com/devnet/flex/articles/itemeditors_pt2.html

Read More

C Version of Adobe Flex Policy Server

Here is a rewrite of the adobe flex policy server written in C.

FlexPolicyServer

This is useful if you are adding a flex based web interface to an embedded system (something running openwrt). You can use this policy server instead of the ones written in perl and python.

Read More

MINI SA from dunehaven

I just got a mini SA board from dunehven.com. My goal was to write a flex UI for a spectrum analyzer. Somr ething I’ve done at work but unable to demo or discuss. So I’ve been looking for a cheap spectrum analyzer to supply the rf data. I’ve been considering a wi-spy but am unable to bring myself to buying a $200 device for this project. Ebay never gave me joy on finding a cheaper model. I’ve bidded on 2 or 3 models bug gave up at 120 dollars.

Then I found the MiniSA for $50 at dunehaven. Just from the specs it looks like it is better than the Wi-spy. However it is a little bit bulkier because in addition to the $50 SA you must buy a open source .Arduino board (~40). I like this solution better because I wanted to get into the AVR microcontrollers anyway.

First impressions of the MINISA is that the software is pretty minimal compared to what I’ve seen Wi-spy does. Wi-spy gives you templates, Spectrum screen, waterfall graphs, statistical displays and all sorts of daemon software.

MiniSA’s windows only client is a minimal Spectrum screen.

The other problem is that minisa’s usb protocol is not documented. I have email out to the designer hoping he will give me some info. Until then I might try my hand at some USB snooping.

Read More

Outsource the Scaffolding

Outsourcing the scaffolding is an article from a fellow Atlantian discussing how best to oursource code. His thesis is that you should only outsource the the initial version or core product. Once you get into the details where the problem is nolonger simple you need to bring it in house.

I agree and I think this applies to outsourced development even when it is a division of the same company. When the designers and requirement gathers are far away from the developers you are asking for trouble.

Read More

Proposed policy. Get rid of the Drug Micro Monopoly.

Require multiple suppliers for any FDA approved drugs.

This seems like a no brainer to me. One of our problems in the US is the expense of drugs. In other countries with more comprehensive government supplied health care there are price controls. In this country it is considered anti-capitalist to be involved with price controls to the point that our sitting president has not given medicare the right to negotiate lower prices directly with drug companies. The US pays more for drugs than other industrialized nations and thus subsidize drug development that other countries benefit from. The government is already involved by giving drug companies temporary monopolies through our patent system. And without this government involvment their industry would collapse.

Of course drug companies say that they have to do a lot of research to get even one  sucessful drug on the market.  So they need these temporary monopolies to get a decent return on investment.  However there is no way for the markets to determine a fair price because a consumer expects their insurance company to pay whatever it takes to keep them alive.  The status-quo is not unacceptable. Between the micro monopolies and the third party payer system the markets do not have room to operate properly.

I would require multiple suppliers for each drug that gets FDA approval. The research company could auction off rights to manufacture/sell their drug to two other companies so there would be at least three suppliers. This would allow the drug research company to recover some of their investment quickly and industry experts (the researcher’s competitors) should be able to determine the proper value of the drug during the auction phase. We would have to be careful of collusion. (Three companies always buying each others patents but never manufacturing their partner’s drugs). But the situation should be no worse than what we have now and should be a little better.

Read More

Available Data Sources

APIs

  • Mapping Shopping
    • http://www.amazon.com/gp/browse.html/ref=sc_fe_l_2/103-1644811-9832630?%5Fencoding=UTF8&node=12738641&no=3435361
    • http://developer.ebay.com/
    • https://partners.shopping.com/app
    • http://developer.yahoo.com/shopping/
Read More

Tools to develop flex on linux

Sometimes you need to write a debug log when developing flex. The best way to do that in linux is first configure your ~/mm.cfg file with the following:

ErrorReportingEnable=1 TraceOutputFileEnable=1

Then either use the flash player debug plugin or use the standalone debug player. I prefer the standalone because I can launch it from the command line right after a successful mxmlc compile. Both are in the tar file here

The log will by default be written at ~/.macromedia/Flash_Player/Logs/flashlog.txt And no this is current as of 2008-9-11, well after the macromedia/adobe merge. Adobe just hasnt bothered changing the path yet.

Read More

The Hidden Knowledge Mangement system.

I’ve recently read Datawocky’s article on Replacing Email with blogs, wikis and IM. Its has some good ideas however the devil is in the details. The advantage of email is that it is convenient for the creator. If it is not convenient for the creator then they are less likely to create/send the data. It might be correct that the 3 system approach would be easier to retrieve data but if it reduces the amount of electronic data then it only hurts the communications.

I think a better approach would be to have special email addresses that automatically publish the content and responses of the emails themselves. We see this for mailing lists on the internet today. Any email written to low level groups in your organization could be published to an internal website. However this does not solve the multiple document revisions being passed back and forth over email problem. To handle documents that are passed back and forth I would argue that better group editing tools would be better such as google docs but for the intranet company. Nobody likes downloading documents to their local machine unless they are taking it off line.

A little scripting on the email server side could eliminate this issue. Whenever a document is emailed out it could be replaced with a link to an editing tool with the document saved on the server side.

Along with this companies should be smart about how they define email addresses. People on the helpdesk should not their person specific email address to customers. Instead they should give a generic email address that can be reassigned to someone else after the original person leaves for a new job. How oftern have you used your email as a datastore. A person that has been with the company for a decade probably keeps a significant amout of his information in his email. A resource a lot of people never look at after the original person leaves.

Read More

Knowledge Management in Services

Just came back from a competitive bidding event. My company was one of the 5 suppliers that were bidding on replacing the customers test equipment with our equipment. We interviewed everyone from the guys in the back office to the guys in the field and the interviewees scored us on the types of questions we asked.

One item that came up was knowledge management. The customer is generating a huge amout of data, but there is no way for them to compare test results from one customer to the other. Or to compare data from different vendors.

There seems to be a great opportunity here for someone to leverage these issues.

Read More

FlashPlayerTrust on Gnu/Linux

I was getting Security Sandbox violations in my flash log. I was trying to read a local file and to connect to the network. This is for development reasons only. The final version is all web. Anyway to resolve this on my linux system I created a file with the top level paths that ended in .cfg. You can place the file in the dir: /home/{user}/.macromedia/Flash_Player/#Security/FlashPlayerTrust directory. For this to affect all users you put it in this directory. /etc/adobe/FlashPlayerTrust/

Read More

Edward Tufte

Yesterday I went to the Edward Tufte Course that my company sent me on. I’m trying to figure out how I can keep the books.The Class rocked. I was inspired by some of the stuff with SparkLines Got some great ideas for the Dashboard design that we’ve been struggling with. Also learned a few other things. Edward Tufte hates Microsoft, loves the iphone and thinks that a handout beats powerpoint any day. I have to admit he made some great points. For instance.1. Data Throughput is slow during Powerpoint presentations2. Handouts that are engrossing are a good thing. Even if the audience ignores you.3. Handouts can sit there for hours while a powerpoint is on display for only a few minutes. And what better way to unsurp your competitor than by giving the potential client something to read during the competitors boring presentation.Other things I learned. Minimize space dedicated to controls. I must have known this already. I love VI editor because there is little space dedicated to controls. Its all about the content. Where as on most IDEs you are luck to get 25% of the screen used for editing code. Hell on this entry form my text box is luck to be 15% of the screen. So I’ll look for ways to hide controls.That is the Gist of what I learned. I truly enjoyed this class and I should have taken it a year earlier.

Read More

A small trick when developing flex code that talks to hosting server

I often develop my flex code on a machine that is not running a web server. So I have to hardcode the IP of a working server when I need to pull data from the hosting server.

This can be a pain because before you checkin the code you have to remember to change the method of pulling the IP address.

So here is my solution var domain:String=mx.utils.URLUtil.getServerName(url); var port_nbr:int=mx.utils.URLUtil.getPort(url); if (flash.system.Security.sandboxType == “localTrusted”) { Alert.show(“In Development mode:”+domain+”,”+port); webService.wsdl=”http://10.0.0.60:3000/services/wsdl”; } else { webService.wsdl=”http://”+domain+”:”+port+”/services/wsdl”; } Obviously this is for web services but this technique can be used for sockets as well.

Read More

SNMP and Ruby

Working with SNMP can be challenging. If you are a Newbie to SNMP and are trying to do something a little unusual (Writing a Ruby agent that sends snmp traps) you probably have trouble finding examples. So I’ve been fighting SNMP for a few days and here are some notes.

Before you start writing ruby code make sure you can send and recieve snmp messages. In my case I was working with traps. This is how I got it working in Ubuntu.

  1. Install net-snmp (This is the snmp library and its daemons.) also install net-snmp-utils which is a lot of the command line tool. On Ubuntu apt-get install net-snmp ; apt-get install net-snmp-utils should work.
  2. On Ubuntu the /etc/default/snmpd file overrides a lot of the default snmp settings. On my version the SNMPDOPTS variable included '127.0.0.1' as the address list. You can just remove this if you want to recieve snmp traps from other servers. Also change 'TRAPDRUN' to 'yes'. This will start snmptrapd daemon.
  3. Try the following command.snmptrap -v 1 -c public localhost 1.3.6.1.4.1.10300.1.1.1.12 localhost 3 0 '' This should work. We are not using any mibs. Something should be written to your /var/log/syslog

If you still have trouble with snmp you can use tcpdump to make sure you are getting a udp message on port 162. I won’t get into that. Its just a sanity check.

Now for the ruby code. To send a version 1 message from ruby you need the snmp gem. So do your ‘gem install snmp’

Some sample code to do the same call


require 'snmp'
SNMP::Manager.open(:Host => "localhost",:Version => :SNMPv1) do |snmp|
x=snmp.trap_v1(
"1.3.6.1.4.1.10300.1.1.1.12",
'localhost',
:enterpriseSpecific, #Generic Trap Type
0,1234
)

end

Notice the :Host key in the Manager object. This is the address of the manager that you want to send the snmp message to. The localhost in the parameters is the ip or domain name of the agent that is sending the trap.

These are just some quick notes mostly for myself and for others who get stuck. It is probably not enough but there are so few examples of this out there on the net it should be useful.

Read More

Resume

Tommie M. Jones

Email: Resume@atlantageek.com

Summary of Qualification * Well versed in web development. * Strong Unix Background, 10 years of experience * Graduate education and experience in Operations Research and Statistics * Experience with object oriented methodologies. * Technologies: SPSS,Oracle, Apache, CGI, Oracle,XML, MS Access, MySQL, Rails, Unix and Windows 2000/NT, Adobe Flex * Unix Systems: (Linux, Solaris, HP-UX, AIX) * xtensive Infrastructure and Architecture experience Professional Experience Dec.2005 - Current Sunrise Telecom Broadband Group Software engineer & Team Lead Sunrise Telecom develops test equiptment for the telecommunications industry. The Broadband Group is responsible for Cable testing equiptment. I was the first web engineer hired in the Atlanta office. Built Web and Linux group inside the Atlanta office Developed 3 web applications from scratch. Two were in PHP/Cake and 1 was in Ruby on Rails and Adobe Flex Was the architect for 2 additional applications (1 windows and 1 PHP/Cake) that was developed by 4 offshore (China) engineers Largest project was a Rails application that used flex for reporting and ruby servers to communicate to monitoring equipment in a proprietary binary protocol. Also required custom MySQL functions written in C that allowed arithmetic functions on arrays in blobs in MySQL Managed the hiring process of 5 engineers. 3 reported to me. 2 were for a new Linux firmware group meant to takeover a project from a closed office. These 2 did not report to me. My group and I were also responsible for configuring and administering the Wiki, Bugzilla, Subversion and a document search engine. Not only was I involved with development I visited customer sites to get feedback on the products. Feb. 2005 - Dec. 2005 Internet Security Systems Software Developer Developed Build tools for product development team. Oct.2004 - Dec. 2004 Verizon Contractor Developed a web based tool that reported, analyzed and forecast cellphone traffic. Software includes Oracle, Mysql, Apache, Perl and Solaris. Analysis techniques include linear regression and exponential smoothing. March 2002 - Oct. 2004 Retek Senior Associate Scientist On site in London, UK for 4 months. I worked with Sainsbury/Accenture on how to improve their forecasting model performance. Analyzed product selling patterns to identify the best fit statistical forecasting model for their product lines. The models used included Bayesian statistics and causal forecasting.

Architect/Lead Developer of Curve 10.5 written in C++. Supervised and Trained Retek Labs contract software engineers on the RPAS API for Version 11.0 Developing Retail Sales Forecasting system using Multi-dimensional database. Developed forecasting software in C++ on both windows and unix. Pricing Analysis for Best Buy using Regression and Mixed Integer Programming. Developed product configuration software in Java using SWING. Taken a client facing role in the later portion of my career at Retek. Worked at a client’s site in London for four months in 2004. Taken a Scientist role doing price optimization and data analysis for one client. Oct. 2001 - March 2002 IBSS Contract Developer Developing tools for development group in Perl and Unix Shell scripting. Part of Development team for custom manufacturing software. May 2000 - Aug. 2001 SilverPop

Senior Developer Designed and Developed Content Management System (CMS) for a Dynamic Messaging System written in PHP, Perl and Java. Since the CMS was a central component of the product, this required major requirement collection from other developers to make sure the product met its goal. Developed with SQL and Oracle as the database Designed and Developed User/Group/Role management component of the Dynamic Messaging System. Coded Unit Testing for automated testing system. Worked with QA and Product Support to resolve lingering issues. Developed mail tracking system using embedded html messages that tracked when and how email was opened (written in Java) Modified Apache Module written in C to connect to Silverpop’s server application through sockets. Worked with XML and XSL. Developed mainly on Linux operating system. Converted CGI scripts to use mod_perl 1998 - May 2000 ICC/GR Software Senior Developer and Release Engineer System Administrator for Linux, Unix, Sun, Solaris, HP and HP-UX systems Maintained last release of MES (Manufacturing Execution System) product written in C, C++ and Oracle SQL. Developed Intranet site that generated dynamic images with Perl/GD that graphed data from bug tracking database (Written in Oracle). Worked with development group to simplify and automate release procedures for main product Wrote Business case for the implementation of a Knowledge Management system. Wrote Several Unix tools in Perl and ksh, that monitored mission critical processes Worked with main development group to simplify release processes. Worked closely with Customers to make sure the product worked on their configuration of Unix/ Database versions. Maintained Infrastructure for development group Developed configurable Proxy server for SFDM’s API written in C, LEX, YACC Converted code to New versions of HP-UX and Solaris and new version of Oracle Acted as Unix System Administrator and DBA for product development team 1995 - 1998 IBSS Developer/Team Lead Worked with AIX, HP-UX and NCR versions of Unix part of development team for proprietary application server that used a lot of IPC functions (Shared memory, semaphores) Interfaced barcode scanners and barcode printers to Unix systems. Used ODBC to interface system to Relational Database. Assisted in design of custom manufacturing floor tracking application On Developmentent team for 4GL compiler project which used LEX & YACC. Team Lead of small team that customized Manufacturing system for customer. Duties included making sure project was on time and interfacing with clients. Worked closely with Customers to solve issues and do training. Maintained order processing system that included manual order entry and EDI Maintained Project written in Perl Skills Operating Systems:

Unix, Linux

Software SAS, SPSS, WordNet, Apache, Internet Explorer, Netscape Navigator, Adobe Photoshop, Microsoft Office

Programming, Protocols and Mark-Up Languages: C, C++, 4GL, SMTP, Perl {LWP, GD}, CGI, DBI, Javascript, Java, HTML, JDBC, XML, LDAP

Databases:

Oracle, MySQL, Microsoft Access

Education 2001 - 2005: Georgia State University, Atlanta Ga Masters of Science in Business Analytics [Robinson School of Business]

1993 - 1995: North Carolina State University Raleigh, NC Masters of Engineering Major: Integrated Manufacturing Systems

1989 - 1993: North Carolina State University Raleigh, NC Bachelor of Science Major: Computer Science Minor: Industrial Engineering

References References available upon request

Read More

Areaware Mango Wooden Radio

A really cool looking design.  Apparently the goal was to create stuff that traditional craftsmen using sustainable materials to create high value products.  Singgih Kartono came up with the Areaware Mango Wooden Radio . I like the idea but I thing a clock design would have carried over better.  I have trouble dedicating that much space to an object I rarely need to look at from afar.  By the way why someone needs to mix a cool wall clock with an mp3 player.  Now that would rock.

Read More

Sun Buys Mysql

Excellent news. Sun acquires Mysql.

MySQL AB :: Kaj Arnö

Three Reasons why Sun bought MySQL

First: Sun has seen its technology base crumble with the rise of Microsoft on the enterprise site and Linux on the Anti-Microsoft side. They’re getting their ass kicked. Java has helped them slow their decline but even there they are not the only Java tool shop in town. Now with MySQL they have a chance. A while back they did a deal with Oracle where they sold a DB appliance with Oracle being the DB. I never saw one being used but it seemed like a great concept. The problem with these DBs is that they are a pain in the ass to tune to your hardware. With the appliance the DB will already be tuned to the hardware.

Second Sun gets to capitalize on its enterprise good name. MySQL is an excellent product but risk averse companies do not consider it an option. With sun’s name behind MySQL it will get an air of legitimacy. So Sun gets to add value to MySQL with very little effort and gets to reap the rewards of the legitimacy with little effort.

Last of all

Sun gets it sales force back in the Web based companies contact list. All these companies trying to be corporate citizens are sending Sun their $500 to pay for MySQL and while they are on the phone they get to hear about Sun’s new hardware lineup. Really when was the last time you went to Sun instead of Dell for your web server needs.

Read More

Visualizing Data by Ben Fry

I just got this book a few days ago and I’m not really that impressed. It seems it is focusing on a Java tool called ‘Processing’ that. He spends more time on how to write code for this tool than on designing data visualization interfaces.

This is unfortunate, I was hoping for a Tufte type book updated to the interactive environment we have today. That would be something useful. Instead I get a tool manual. I still have 2/3 of the book left. Hopefully it improves.

Read More

Zed's Rant.

I just read Zed’s rant. (Zed is the developer of Mongrel, a recent & popular web server written in Ruby) Interesting stuff. Not just because he obviously has an Axe to grind but he does give a look at all the stuff going on behind the scenes with Ruby and Rails.

One item that caught my interest was the assertion that Japanese developers ignoring patch submissions from developers. Ruby does have some major performance issues. There is a reason there are at least 2 virtual machines (Rubinus and JRuby) being developed to compete with the current Ruby. And I do wonder if part of the problem is the (spoken) language barrier between the core team and a good chunk of the user market.

Maybe it is time for the language to branch.

Read More

The Elements of online Journalism

One of the fringe benefits of married life is having someone to get you those Christmas gifts you can’t bring your self to buy. And for me those gifts are books. And the week between Christmas and New Year is the best time to read them.

One of my books is “The Elements of Online Journalism” by Rey G. Rosales. Its a nice little overpriced pamphlet of a book topping off at 50 pages. Its targeted to existing journalist who are interested in online journalism. Rosales suggests that there is more to online journalism than just reprinting new paper articles and photos. He talks a lot about flash animations that allow the reader to view and explore data.

This is the type of stuff I’m interested in. Why aren’t there more examples of this out there. There are a few apps out there that do a good job of this. For the last presidential election one newspaper generate a flash animation that let you play what if scenarios with the electoral college. The app would categorize states in to 5 groups depending on how likely Bush or Kerry was to win the state with an undecided group right in the middle. The user could assign electoral votes to one president or another and see how close the candidate was winning. A use could get a much better understanding of the race than any news paper article could give them.

So why are there not more of these animations. Where is the animation showing the time line of the Iraq and Afghanistan war. Or showing how the housing boom busted. Why is there not a company that all it does is generate these animations and then rent or sell them to newspapers and other media outlets. A few mentions on Slashdot or digg should pay for the operating cost of a couple of Indian programmers and a couple of guys in the states designing and selling the ideas.

I have two more books to read “Flash Journalism” by Mindy McAdams and Visualizing Data by Ben Fry. Flash Journalism gives more examples of the journalistic flash animations mentioned above. Visualizing data is about communicating data with graphs. Another excellent read.

Read More