18 December 2010

Can't SSH to Amazon Elastic Compute Cloud (EC2) Instance

I've decided that I'm going to learn about Amazon Elastic Compute Cloud (EC2) over this Holiday break.  I've read a few articles, listened to a few podcasts so I have an idea what it's about, just haven't actually gone thru the motions myself.

What am I going to do with a virtual machine (VM) in the cloud?  Not much quite yet.  I have some ideas but first need to figure out how things work.

So on Friday night I signed up for an EC2 account and started to follow the Getting Started Guide on the Amazon site.  As usual, things went smooth for the first few minutes but as usual, progress came to an immediate screeching.

I started an plain vanilla Basic 64-bit Amazon Linux Amazon Machine Image (AMI) just to get things going and verify that I could SSH into the thing per the getting started guide.  But when I SSH'd, nothing, nada, zip.  Timeout.


This darn thing acts like it's not running.  I started, restarted, used different AMI, mucked with security group setting without any progress.  I even edited the default security group to open the thing wide open:


Still nothing.  Did the google thing to see what other people are doing and came across a few hits in the Amazon Web Services forums where people are claiming the the web services aren't starting up correctly in the US-East zone.  Hey, that's me!  Ok, I'm not stupid, its just some tech problem.  I'll take a break and try it again in a few hours.

Few hours later:  nothing, nada, zip.  Ok, I am stupid.  I can't even log into a stupid instance.  I probably started/stopped an EC2 AMI 10 times but can't do anything with it.  This new thing I've learned is frickin' amazing.

And as usual, as soon as I've convinced myself that all hope is lost, ding.  Wait, doesn't my Westell DSL modem block SSH?  Oh, frick of course it does.  Damn it so, the last 3-4 hours of trouble was caused by my own firewall??

Of course it was.  I really hate this Verizon provided modem and keep telling myself that I'm going to buy a different one.  But this is free and I'm cheap.  So I'll keep shooting myself in the foot and someday get smarter and dump this junk.

In the Firewall->Port Forwarding menu, I just added the SSH service to a dynamic host.


And now when I launch a new instance, I get this:


Amazing.  Only hours and hours of work to launch and connect.  Let the fun begin.

What do you think? Leave a comment.

29 November 2010

Working with the 90 Character Limit of Twitter

What?  Twitter has a 90 character limit?  No, it's 140.  Everyone knows that.

Mr Wikipedia says this about SMS length

"... resources in the system could be used to transport messages at minimal cost. However, it was necessary to limit the length of the messages to 128 bytes (later improved to 140 bytes, or 160 seven-bit characters) so that the messages could fit into the existing signaling formats."

Using the Groovy code that I did the other day to send a text message via a twitter Direct Message, if the message length is greater than 140 characters, twitter complains with this error:

403:The request is understood, but it has been refused.  An accompanying error message will explain why.
{"request":"\/1\/direct_messages\/new.json","error":"The text of your direct message is over 140 characters."}
TwitterException{exceptionCode=[6bcd7469-01bff100], statusCode=403, retryAfter=0, rateLimitStatus=null, version=2.1.7}


That kinda says that I can send 140 characters, so what's this nonsense about a 90 character limit?

Well, true, you can send 140 characters but if you're hacking the system for the purposes of having Twitter forward your message as a text message to a phone then different rules apply.

Take a look at what happens when you send a Direct Message of 140 characters to an account that is set up to send a txt:
String message = "12345678911234567892123456789312345678941234567895" +
                 "12345678961234567897123456789812345678991234567891" +
                 "1234567891123456789212345678931234567894"

DirectMessage directMessage = twitter.sendDirectMessage(screenName, message)
Twitter breaks the message into 2 separate texts (and my phone occasionally gets the 2nd text first).

For the app that I'm working on, I need my data in a single text message, I don't want the mess of re-assembling.

I think the message split is because a header of:

"Direct from" screen name:

and a footer of

"Reply with 'd" screen name "hi.'"

is added to each message.

I've found that the amount of data that you can actually send depends on the screen name.  Since screen names can be up to 15 chars, you lose:

Header:
Direct from screenname:
= up to 28 characters (12 + screen name)

Footer:
Reply with 'd screenname hi.'
= up to 34 characters (19 + screen name)

140 - 28 - 34 = 78

Uh, 78 is not 90.  What the heck? I guess my math has errors.

But check this out.  By adjusting my direct message, I find that a 90 character message is the sweet spot for a 15 character screen name and the message is sent as a single text.  91 characters will cause the message to be split over 2 texts.

Maybe that header and footer isn't as big as I think it is.

140 - 90 = 50 characters for header/footer?

If a screen name is used twice (2 * 15 = 30), then maybe the bloat is only 20 characters?


What happens if I send messages to a different screen name, one that is smaller than 15 characters?

What is I use a screen name of 8 characters?  Should I be able to send a message of

140 - 20 (bloat) - (2 * 8) = 104 characters?

After a quick reassignment of my phone to a different twitter account, I tried a test with a screen name that is 8 characters.  Could I send 104?  Durn right I could!  And I could also send 109.  But 110 would cause a message split.  So I have an extra 5 characters unaccounted  for.

What happens if I send to a ...  No, this is going on too long. 

The max screen name is 15 characters and that limits the max amount of data that can be sent in one message to 90 characters.

If the screen name is less than 15 characters, then you get 90 plus some extra.  YMMV


What do you think? Leave a comment.

27 November 2010

Send SMS Text Message from Java/Groovy - via Twitter

Writing code to send an SMS text message is fairly easy on a smart phone like an Android.  But from a desktop, sending texts is still a mess, not always free, and/or requires a phone connected to your server.

I simply want to send a message to my phone when my app receives some data.  I don't need a complex library with lots of setup nor do I want to sign up for a pay service.

Now this isn't the cleanest solution to text from an app but Twitter can send messages to your phone when people you follow update their status, when someone mentions or replies to you, or when you get a direct message.

My last two write-ups, "Send a Simple Tweet with OAuth and Groovy" and "Send a Direct Message Tweet with Groovy" showed how easy it is to use Java/Groovy and the Twitter4J library to send messages to your own Twitter account or a message to anyone else.

By sending a tweet to an account that you follow or a Direct Message to your own account, Twitter can be setup to forward that message to a phone that you associate with your account.

Simply go to your Twitter account, Settings, Mobile and follow the easy instruction. Then use one of the sample apps from my previous posts and tweet-text away!

Note:
  • This only allows you to tweet-text to phones that have been setup to follow accounts. 
  • This does not allow you to send a text to a cell phone not registered with Twitter.
  • This works for what I need.
  • Do you know a better/easier, free method?  Write a comment!


What do you think? Leave a comment.

Send a Direct Message Tweet with Groovy

Alright.  Using the Twitter4J library and Groovy, I've beaten down the OAuth beast and have a simple example app to send a "update status" tweet.

Now instead of sending a message to my own account, let's do a real easy mod to send a Direct Message to a specific user   Using the same app as before, the only changes needed are:

  • import DirectMessage
  • change from twitter.updateStatus(message) to twitter.sendDirectMessage(screenName, message)
import twitter4j.DirectMessage;
import twitter4j.Twitter
import twitter4j.TwitterFactory
import twitter4j.Status
import twitter4j.http.AccessToken
import twitter4j.http.RequestToken
import java.io.BufferedReader
import com.thoughtworks.xstream.XStream

try{

   Twitter twitter = new TwitterFactory().getInstance();

   // set key and secret that you get from Twitter app registeration at:
   //     http://dev.twitter.com/pages/auth#register
   twitter.setOAuthConsumer("Your Consumer key", "Your Consumer secret");

   // load access token if it exists
   AccessToken accessToken = null
   def tokenFile = new File("accessToken.xml")

   if (tokenFile.exists()) {
      def xstream = new XStream()
      tokenFile.withInputStream { ins -> accessToken = xstream.fromXML(ins) }
      twitter.setOAuthAccessToken(accessToken)
   } 

   else {

     // get the URL to request access to Twitter acct
     RequestToken requestToken = twitter.getOAuthRequestToken();
     String authUrl = requestToken.getAuthorizationURL()
     System.out.println("Open the following URL and grant access to your account:");
     System.out.println(authUrl);

     // take the PIN and get access token
     System.out.print("Enter the PIN:");
     BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
     String pin = ""
     pin = br.readLine();
     accessToken = twitter.getOAuthAccessToken(requestToken, pin);

     // persist token
     def xstream = new XStream()
     xstream.classLoader = getClass().classLoader
     new File("accessToken.xml").withOutputStream { out -> xstream.toXML(accessToken, out) }
   }

   String message = "from Groovy w/ token" + accessToken.getToken()
   String screenName ="ScreenNameToSendTo"

   DirectMessage directMessage = twitter.sendDirectMessage(screenName, message)
   System.out.println("Direct message sent to " + directMessage.getRecipientScreenName());

} catch (Exception e) {
   e.printStackTrace();
}

What do you think? Leave a comment.

Sending a Simple Tweet with OAuth and Java/Groovy

Come on, man!  What used to a be a simple task to send a tweet is now a OAuth nightmare.

A year or so ago, I built that tweet-a-w/e thing that sniffed XBee chirps and sent them to a twitter account that kindly routed them to my cell phone.

I have a need now to receive an XML stream, parse out a few tidbits and then send the results out as a SMS text message.  Remembering that tweet-a-w/e app, I thought that I would again leverage Twitter to send the text message.  All I need to do is send a tweet via a Twitter API and have my account set up to send the content to a phone number.

This time my XML receiver is in Java/Groovy, so I grabbed the latest Twitter4J, glanced at the UpdateStatus example and tried a quick test with Groovy.
import twitter4j.Twitter  
import twitter4j.TwitterFactory  
import twitter4j.Status  
    
Twitter twitter = new TwitterFactory().getInstance("myAcct","myPassword");  
Status status = twitter.updateStatus("from Groovy");  
System.out.println("Successfully updated the status to [" + status.getText() + "].");  
Did that work?  Of course not.  Authentication FAIL.

Caught: 401:Authentication credentials were missing or incorrect.
{"errors":[{"code":53,"message":"Basic authentication is not supported"}]}
TwitterException{exceptionCode=[15bb6564-00e5bee0], statusCode=401, retryAfter=0, rateLimitStatus=null, version=2.1.7}

What the heck?  Basic Authentication is not supported?  Darn it, what is this stupid OAuth thing about?  Here's the Twitter page "Authenticating Requests with OAuth" that looks real interesting to read.  Arghhhhh.  I just want to send a flippin' tweet, I really don't have time to research this at Hueniverse.

After some more quick Googling, this isn't as bad as it first looks.It boils down to getting a key and secret pair.  The steps are:
  • Goto Twitter and register your app.
  • You'll get a consumer key and secret which are similar to the public and private keys used in protocols such as ssh. 
  • Use the key and secret to sign every request you make to the Twitter API
import twitter4j.Twitter  
import twitter4j.Twitter
import twitter4j.TwitterFactory
import twitter4j.Status
import twitter4j.http.AccessToken
import twitter4j.http.RequestToken
import java.io.BufferedReader

try{

   Twitter twitter = new TwitterFactory().getInstance();

   // set key and secret that you get from Twitter app registeration at:
   //     http://dev.twitter.com/pages/auth#register
   twitter.setOAuthConsumer("Your Consumer key", "Your Consumer secret");

   // get the URL to request access to Twitter acct
   RequestToken requestToken = twitter.getOAuthRequestToken();
   String authUrl = requestToken.getAuthorizationURL()
   System.out.println("Open the following URL and grant access to your account:");
   System.out.println(authUrl);

   // take the PIN and get access token
   System.out.print("Enter the PIN:");
   BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
   String pin = ""
   pin = br.readLine();
   AccessToken accessToken = twitter.getOAuthAccessToken(requestToken, pin);

   String message = "from Groovy w/ pin" + pin
   Status status = twitter.updateStatus(message);
   System.out.println("Successfully sent " + message);

} catch (Exception e) {
   e.printStackTrace();
} 

Ok, so that works.  But every time you run the program, you need to go and fetch the darn token.  According to Twitter, the token doesn't expire, so let's persist the thing.

I like using XStream to serialize Java/Groovy objects to/from XML.  A quick update to the code above lets us save an XML file that we can reload as needed.
import twitter4j.Twitter
import twitter4j.TwitterFactory
import twitter4j.Status
import twitter4j.http.AccessToken
import twitter4j.http.RequestToken
import java.io.BufferedReader
import com.thoughtworks.xstream.XStream

try{

   Twitter twitter = new TwitterFactory().getInstance();

   // set key and secret that you get from Twitter app registeration at:
   //     http://dev.twitter.com/pages/auth#register
   twitter.setOAuthConsumer("Your Consumer key", "Your Consumer secret");

   // load access token if it exists
   AccessToken accessToken = null
   def tokenFile = new File("accessToken.xml")

   if (tokenFile.exists()) {
      def xstream = new XStream()
      tokenFile.withInputStream { ins -> accessToken = xstream.fromXML(ins) }
      twitter.setOAuthAccessToken(accessToken)
   } 

   else {

     // get the URL to request access to Twitter acct
     RequestToken requestToken = twitter.getOAuthRequestToken();
     String authUrl = requestToken.getAuthorizationURL()
     System.out.println("Open the following URL and grant access to your account:");
     System.out.println(authUrl);

     // take the PIN and get access token
     System.out.print("Enter the PIN:");
     BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
     String pin = ""
     pin = br.readLine();
     accessToken = twitter.getOAuthAccessToken(requestToken, pin);

     // persist token
     def xstream = new XStream()
     xstream.classLoader = getClass().classLoader
     new File("accessToken.xml").withOutputStream { out -> xstream.toXML(accessToken, out) }
   }

   String message = "from Groovy w/ token" + accessToken.getToken()
   Status status = twitter.updateStatus(message);
   System.out.println("Successfully sent " + message);

} catch (Exception e) {
   e.printStackTrace();
}

The serialized Access Token looks like this:


  5555555-AbCdEF
  1oKiOlOlOlOl
  
    oauth_token=5555555-AbCdEF
    oauth_token_secret=1oKiOlOlOlOl
    user_id=007
    screen_name=bond

  
  bond
  007

Notes:

For the above examples, I used:
  • Groovy Version: 1.7.0 JVM: 1.6.0_22
  • Twitter4J 2.1.7  For this, I copied only the twitter4j-core-2.1.7.jar into the Groovy lib folder.  On Linux, this is /usr/share/Groovy/lib
  • XStream 1.3.1  For this, I copied the xstream-1.3.1.jar and xpp3_min-1.1.4c.jar into the Groovy lib
What do you think? Leave a comment.

13 August 2010

Sad Day for the Great White Boxster

After a little more than a year of ownership and the main reason that I haven't been updating my blog, I sold my Boxster (the Great White). She was a good car but after buying a 911 in early Spring, I just wasn't driving her much.

My family is much sadder than me though cause they liked the Boxster better than the 911.

I really love the mid-engine Boxster and will probably buy a Cayman S in the near future.  After getting the Porsche bug, I wasn't happy until I had a 911.  What a complete different experience.  The Boxster is refined and hugs the curves like no other car I ever driven.  The 911 is raw, loud and dares you to give it too much gas so it can swing its ass around and make you look like a noob driver.  But it's so fast and the power is addicting.

09 August 2010

Sanyo DP26648 White Screen of Death

 Santa brought my game player son a Sanyo SP2664 26-inch Widescreen LCD HD Television last Christmas.  A few months into the year, the TV started getting occasional white lines diagonally across the screen, the greens looked weird, and then the audio went in/out.  I could've/should've hauled it back to Wally Mart but grumbled about it and did nothing.

We came home from our July 4th vacation and the darn TV is dead.  White Screen of death.  No sound, just a stupid looking white screen.

An interesting thread on the Electronics Repair Google Group led my son and I to think that we could fix this beast.

According to the thread, "it has been reported that there is a defective ribbon cable that causes this problem".  And the possible fix is "Clean glue like material from spaced further apart trace of FFC ribbon or replace the ribbon wire P/N: N6CD FFC".

Well, I'm pleased to report that we hacked and smashed and tore that darn TV up, finally got to the elusive cable.  And it looks nothing like that picture.  Our cable looked fine, no glue, seemed like a good connection.  And the TV remains broken and in 300-bazillion parts.

We had fun exploring the TV but it would have been a better story if the darn thing would have actually been easy to fix.  I think our TV had more wrong with it than just the cable.  But if your TV is showing the white screen of death, read the link and good luck.

25 December 2009

LG Washer LE Error

UPDATE:  I've had quite a few hits on this posting and received lots of great comments - thanks very much.

This post explains how to replace the hall sensor on your LG washer.  Before you decide that you have to replace your hall sensor, please try unplugging your washer for 5-10 minutes and see if it will reset all on its own.

Also, even after replacing the sensor, sometimes these washers will display "LE" a week or month later and they just need a time-out (unplugged).  Good luck!

I've also had a few questions on what the heck is a hall sensor. Here's a whitepaper with a good description, design notes and ideas on what else you can do with these things.

-------------------------------------------------------------------------

We had a really nice Maytag washer and dryer set. They were a billion years old (maybe 15 actually) but ran strong and had zero problems. Spring 2007 we decided to remodel our laundry room and of course we had to have the super duper LG WM2075CW 27" Front-Load Washer with 3.72 cu. ft. Capacity, 7 Cycles, 6 Options, SenseClean and 8 Hours Delay Wash.

About $800 delivered. Nothing but the best for my socks and undies (boxers btw).

Everything has been great except the week before Christmas 2009, Mr LG wouldn't work. It would grunt and click and then put "LE" on the display. A quick Google yields lots and lots and lots of hits with the Load Error (LE).

A number of things can cause an LE but the most common is a malfunctioning Hall Sensor. Seems that LG washers are pieces of crap and this is a well known problem. LG has extended the warranty on the part to 7 years but they won't ship the in-warranty part to consumers. They'll only ship to a certified service company. After several phone calls to LG and way too many minutes on hold, I finally just ordered the damn thing from MCM Electronics, part number 6501KW2002A for $18. Plus tax and shipping, at my door for $29.75.

Actually the LG washer and dryer have been great, I'm just very ticked that LG has a known problem but won't do a recall to fix the part with a more reliable piece. And even more ticked that they won't ship me an in-warranty part so I can do it myself. So for now on, I will refer to the washer as a piece of crap and LG as a seller of crap. My next washer will be a Bosch.


The Google hit that explained the error, the part number and how to replace is FixYa. Their how-to was a little brief and a picture is worth a thousand so I thought I would help out the next sucker that also has a piece of crap that needs to be fixed.

Read thru that FixYa article and make a decision if your problem is also the Hall Sensor. If so, follow along.

This is a pretty simple fix. If you can change a sparkplug than you can easily do this one. (I compared this to a sparkplug change cause you'll get a little dirty, gotta pull some cables, and work a socket wrench).

You'll need a phillips head screw driver, 17mm and 10mm sockets or wrenches.

A quick note:  Even though the pictures are somehow dated 1/7/2006, they were actually taken in December 2009.  Someone reset my camera date and I didn't notice until pics were all taken.

Ok, First step, UNPLUG the washer. And carefully move the washer so that you can access the back panel. If your washer and dryer are stacked, please be careful and make sure there is nothing on top that will fall and hit you in the head. Ask someone to help you, don't blame me if you hurt your self moving the thing. Be careful!
  • Remove the 4 phillips head screws on the back panel
  • With a 17mm socket, remove the bolt that holds on the motor rotor. Turn the bolt to the left (counter clockwise) to remove. The drum will probably turn as you're trying to get the bolt loose so with your free hand, grab the rotor around 10 o'clock and hold tight while you turn the bolt. Be patient and it'll come right out.
  • Next,you need to pull the motor rotor off to expose the stator. This will be a little tough because of the magnets and a tight fit. Use both hands on opposite sides and gently tug one side, then the other, then both. Try your hands at 1 o'clock and 7 o'clock. Then switch to 5 and 11. Be patient and tug gently. Watch the white center and you'll see it slowly pull away from the shaft.


  • Set the rotor to the side and remove the 10mm bolts holding the stator on. I wasn't sure what to expect when I started removing it and I wanted to make sure I put it on right side up so I made pencil marks to line things back up. These really aren't needed cause it'll be obvious as to which side is up.
  • When you remove the 6th bolt, use your free hand to hold on to the stator cause it will fall right off. There are two connectors at the bottom, one the goes to the motor windings, the other to the hall sensor that we're replacing. If you don't hold onto the stator, you might damage the wires or connectors. Go slow on the last bolt and hang on.
  • Here's what those connector look like from the bottom side. I didn't have any luck trying to get them free from the bottom though. I was kinda squeezed in between the wall and washer so I didn't have much room to wiggle. I let the stator lean towards me like the picture above and then gently pulled both connectors free. Don't go Rambo on these connectors, you don't need to replace extra stuff. Breathe deep and squeeze the little tabs, they'll come free.
  • Almost there. That shiny looking thing on the bottom right of the stator ring is the hall sensor. It's held on with 3 clips on the back side, 1 clip on the front. Simple pry up the 1 clip and it will pop free.
  • The new sensor snaps back on just like you'd expect. Hook up the 3 clips first and then snap it on to the 1 clip side.
  • I don't have pictures of these next steps cause it's just the opposite of the above. Just look at the pictures in the reverse order and it'll all go right back together.
  • Hook up the two connectors. Make sure they click all the way back on.
  • Line the stator back up with the bolt holes and screw the bolts back in with a 10mm socket. Tighten them firm but don't over do it.
  • Push the rotor back onto the shaft. Because of the magnets and the shaft grooves, it will feel like it's not going. It will, line it up and apply pressure at the edges.
  • Tighten the big bolt with a 17mm socket. As it tightens, the drum will start to rotate. Grab the edge with your free hand and tighten firmly.
  • Put the back panel back onand tighten down the 4 phillips head screws.
  • Plug the washer back in and carefully push the washer back into place.
  • Test.
  • Your piece of crap washer is fixed.
Total time was 38 minutes. If I didn't stop and take pictures, easily under 30 minutes.

What do you think? Leave a comment, buy me a beer!



15 May 2009

Porsche Boxster Cabriolet

Sometimes you just have to give in. I tried to be strong but night after night. Day after day. All I heard was "buy me, buy me". How much is a man expected to take? We can only be so strong before we all break down.

It was hard, but I relented and came home with a Boxster. It's my combo birthday present and mid-life crisis consolation prize.

As my old age time bomb has been approaching over the last year, I've been thinking about the 40th anniversary Camaro.

I had a 1969 Camaro in my Senior year of high school. During the first semester, a girl in my first period class was driving her brother's 1965 Mustang to school and somehow drove her front bumper all the way into my backseat. And then tried to hit-n-run. Since the cars had become melded into one, she didn't get very far but she sure did try.

Anyway, I've been thinking a lot about the new Camaros. But with all the trouble GM has been having, I'm just not excited about paying $10K premium to a company that's on the verge of bankrupcy. And the last few years of Chevrolets really haven't seemed like quality cars. Probably not a good choice.

So I've been cruising Craigslist and eBay looking at other sports cars that I've wanted since I was 13. High on the list was a Datsun 280Z. Man, I wanted one of those so bad back then. When I look at them now, they're kinda cheesy. Not sure what was driving my desire back then. Maybe puberty.

Last week, I thought I had found the car. A Porsche Boxster Cabriolet. Price was right. Bank said yes. And boss said yes. Yes! No. The owner said it had been in an accident and the hood was replaced. That's not bad really. I mean as long as nothing was seriously crunched, for the right price, that wouldn't bother me. The owner said unless he hadn't told me, I'd never know. Said it was a perfect match/fit and there was no lingering problems. Then the night before I was to drive to Chicago to see it, it was disclosed that the frame "was bent or had been bent". What? I talked to the guy a dozen times, emailed a dozen more. And the night before I see it, it's "disclosed"? No way, wasn't going to waste any more time on that dream.

The idea of a Boxster Roadster stayed with me though. And Mr Craiglist led me to the next beautiful car. The Great White.

One word to sum up the car. Wow.

I love it. It is so clean. Perfect interior and exterior. Looks brand new. Nice throaty rumble, comfy leather seats, decent audio. Drives so nice.

When I told the kids that I spent their college funds and they'd have to attend community colleges now, my daughter said. "That's okay. Will you drive me to school tomorrow?"



Update: Sad Day for the Great White Boxster

02 May 2009

Greasemonkey Index of /images

After I posted the Yahoo Baseball Greasemonkey script, I remembered another script that I have that's worth sharing. Not my Yahoo Football Greasemonkey script which does the same as the baseball (shows in-game stats on tab title) but another more useless useful script.

Have you ever been searching for an image or been Stumbling around and came across a page like the one to the left?

I see them quite often actually because I'm nosy and like to poke around on web sites. If I see a picture of interest, I wonder if the web site owner has others that aren't being displayed.

But here's the rub. We have a nice list but we have to click each one to see it.

Greasemonkey to the rescue.

I wrote a quick script that replaces the little icon to the left of each link with the link src. And since loading these images may be time consuming, you don't want to do this on every "Index of /" page that you come across. So we wrap that function up in a "Show Images" link at the top of the page when it completes loading.

I've uploaded the script to userscripts.org where you can check it out. Install it and go to a page where you can see for yourself how it works.

Google around for "index of /images" and see what treasures abound.

I find lots of pages like this by "going up" a level on a URL. For example (not real, this is not a valid location) if you are looking at a photo:

http://somewhere.good/images/138028.jpg

Then see what's at:

http://somewhere.good/images

Either by editing the URL in the address bar or use the "up one level on this web site" button in Firefox. What, you don't know about that button? Right-Click an empty spot the tool bar and select "Customize...". Drag the icon that looks like the one I've circled below to your tool bar. Use it and discover lots of stuff.


What do you think? Leave a comment.

Baseball Better - Yahoo Greasemonkey Script

I love baseball. But you gotta admit that it can be boring "watching" a game on the web. If not watching it in person or on tv, it's kinda slow. I'm sorta ADD when cruising the web. I usually have 6 or more Firefox tabs open at once. I'm reading something, writing something, searching for something all at the same time. It's a wonder that I get anything done at all, but amazingly it works out.

So if a good game is on at the same time that I'm on-line, I like to have a tab tuned into the Yahoo Sport MLB game. But the problem is that I have to keep clicking to that tab to see what's going on. I have Greasemonkey installed and have written a few other scripts to customize my viewing pleasure so hacking the box score page should be pretty easy.

Here's what the regular game looks like. Usually there's an advertisement over on the right side above the "series at glance" table. I guess times are tough at Yahoo and advertising is slow because no ad is being displayed on this screen shot.

Notice a few other things. The Firefox tab says "MLB - Kansas City Royals/M...". I'd like this to display something useful like the current in-progress game stats. Also see the Inning Summary table at the bottom. Below that is a Scoring Summary that can't be seen unless you scroll down.

So my typical game "watching" experience is every 5-10 minutes, click the tab, look at the score. If different than last time I looked, grab the scroll bar and go way down to see what happened. Way too much work.

After some quick Greasemonkey hacking, I've come up with a better experience. As the page loads, I grab the team names, current score, inning, ball-strike count, runners on-base, and last play. Sounds like a lot but really it's not. Yahoo has everything in nice tables and getting it is like butter. Not really but I just felt like using that term. Just like buddah.

I take all those stats and format them into the tab so I can see the game info without having to click the tab. This alone has probably saved years of wear and tear on my mouse button.

I then replaced the advertisement block (yahoo isn't currently using it anyway) with the scoring summary so no painful scrolling occurs.

For those that also want to enjoy baseball a little better, I've uploaded the script to Greasemonkey script haven.

For those that don't trust me and want to see what I've done, here's a view in the sausage factory. It ain't pretty but it works.

It's just basically screen scraping by searching for classes and then parsing data. Every time Yahoo makes a tweak, it breaks the script. I don't mind, it's usually minor.

Here we go.

Yahoo sets the page to a fixed width of 974 pixels. I run in 1280 mode so this wastes lots of valuable screen space and causes unsightly white gutters. This must be changed:
function changeWidth() {
var node = document.evaluate('//table[@width="974"]',
document,
null,
XPathResult.UNORDERED_NODE_SNAPSHOT_TYPE,
null);

node.snapshotItem(0).width="100%";
}
Here's a big part of the magic. Find the table that wraps the team name and score and parse it out. The team names are long and we need shorten them so they fit it the tab nicely.
function getScore() {
var scoreNode = document.evaluate(
'//td[@class="yspsctnhdln"]',
document,
null,
XPathResult.UNORDERED_NODE_SNAPSHOT_TYPE,
null);
scoreNode = (scoreNode.snapshotItem(0));

// this is ugly but I want to shorten the team so all the info displays nice on the tab
var str = scoreNode.textContent;
str = str.replace(/\n/g, "");
str = str.replace(/ /g, "");
str = str.replace(/,/g, "");
str = str.replace("Arizona", "AZ");
str = str.replace("Atlanta", "Atl");
str = str.replace("Baltimore", "Bal");
str = str.replace("Boston", "Bos");
str = str.replace("Chi Cubs", "Cub");
str = str.replace("Chi White Sox", "CWS");
str = str.replace("Cincinnati", "Cin");
str = str.replace("Cleveland", "Cle");
str = str.replace("Colorado", "Col");
str = str.replace("Detroit", "Det");
str = str.replace("Florida", "FL");
str = str.replace("Houston", "Hou");
str = str.replace("Kansas City", "KC");
str = str.replace("Minnesota", "Min");
str = str.replace("Milwaukee", "Mil");
str = str.replace("LA Angels", "LAA");
str = str.replace("LA Dodgers", "LAD");
str = str.replace("NY Mets", "Met");
str = str.replace("NY Yankees", "NYY");
str = str.replace("Oakland", "Oak");
str = str.replace("Philadelphia", "Phi");
str = str.replace("Pittsburgh", "Pit");
str = str.replace("San Diego", "SD");
str = str.replace("San Francisco", "SF");
str = str.replace("Seattle", "Sea");
str = str.replace("St. Louis", "SL");
str = str.replace("Tampa Bay", "Tam");
str = str.replace("Toronto", "Tor");
str = str.replace("Texas", "Tex");
str = str.replace("Washington", "Was");
str = str.replace(/ /g, "");

// get inning and shorten
var inning = getInning();
inning = inning.replace(/\n/g, "");
inning = inning.replace(/ /g, "");
inning = inning.replace(/Bot /g, "B");
inning = inning.replace(/Top /g, "T");
inning = inning.replace(/End /g, "E");

document.title = str + ' ' + inning + ' ' + getOut();
}
In the above code, I didn't put a comment on this line but this is where the Firefox tab gets the in-game stats:
document.title = str + ' ' + inning + ' ' + getOut();
And more parsing. And then some really ugly if-else code. Kinda embarrassing that I did this. But I did and I'm too lazy to re-write.
function getOut() {
var balls = document.evaluate("//*[contains(.,'O:')]/b",
document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue;

var count = "";
if (balls) {
var n= balls.nextSibling;
var strikes = n.nextSibling;
n = strikes.nextSibling;
var outs= n.nextSibling;
count = balls.innerHTML + '-' + strikes.innerHTML + ' O:'+ outs.innerHTML;
}

// find how many men are on base.
// do this by looking at what image is being displayed.
// yeah, I need to clean this up but it works and I don't have time. go for it
var men = document.evaluate("//img[contains(@src,'tr_empty.gif')]",
document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue;
if (men) {
men = "empty";
}
else {
men = document.evaluate("//img[contains(@src,'tr_1b.gif')]",
document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue;

if (men) {
men = "1b";
}
else {
men = document.evaluate("//img[contains(@src,'tr_2b.gif')]",
document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue;
if (men) {
men = "2b";
}
else {
men = document.evaluate("//img[contains(@src,'tr_3b.gif')]",
document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue;
if (men) {
men = "3b";
}
else {
men = document.evaluate("//img[contains(@src,'tr_1b2b.gif')]",
document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue;
if (men) {
men = "1b2b";
}
else {
men = document.evaluate("//img[contains(@src,'tr_1b3b.gif')]",
document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue;
if (men) {
men = "1b3b";
}
else {
men = document.evaluate("//img[contains(@src,'tr_2b3b.gif')]",
document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue;
if (men) {
men = "2b3b";
}
else {
men = document.evaluate("//img[contains(@src,'tr_full.gif')]",
document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue;
if (men) {
men = "full";
}
else {
men = "";
}
}
}
}
}
}
}
}
return count + ' ' + men;
}
And that's about it. Take a look at the file at userscripts.org and give it a try. Let me know if you've improved and found some bugs.

What do you think? Leave a comment.

08 April 2009

XBee Series 2 - Reading Data in API Mode

After I finally got a ZNET 2.5 Coordinator AT associated with a ZNET 2.5 ROUTER / END DEVICE by figuring out that the default JV Channel Verification parameter was key, I stumbled again.

I can send character data from end device to coordinator and see it come out of the UART.

But when I have an end device with the ADC enabled (ATD1=2) and the IO Sampling Rate (ATIR=4E20) set to 20 seconds, I see the coordinator's red LED flicker as data is being received. But nothing appears to come out of the coordinator's UART.

Back to drawing board with Mr Google. I searched websites, blogs, forums. Some are claiming it works great, others are having issues. I'm on the issue side.

Then came across this in the XBee ZNET 2.5 Product Manual, section 4.5.3. Periodic IO Sampling :
Periodic sampling allows an XBee / XBee-PRO module to take an IO sample and transmit it to a remote device at a periodic rate. The periodic sample rate is set by the IR command. If IR is set to 0, periodic sampling is disabled. For all other values of IR, data will be sampled after IR milliseconds have elapsed and transmitted to a remote device. The DH and DL commands determine the destination address of the IO samples. DH and DL can be set to 0 to transmit to the coordinator, or to the 64-bit address of the remote device (SH and SL). Only devices running API firmware can send IO data samples out their Uart. Devices running AT firmware will discard received IO data samples.
I swear that when I searched around, I found people claiming to receive IO data in AT mode. I surely can't so maybe the manual is on to something. I'll try API mode. Bet it's a boatload of fun.

TIP: When writing API firmware to a device that currently has AT firmware on it (or visa-versa), after the write, X-CTU complains that it failed to enter command mode. This is because the "Enable API" setting on the PC Settings / Host Setup tabs need to be toggled.

After putting ZNET 2.5 COORDINATOR API on, I did start to seem to receive data. But not what I expected. With AT Transparent Mode, I could just read a string straight from the UART and parse away. Now in API Mode I'm getting some weird gook that sure doesn't print very nice. Probably that API Frame thing that I skimmed past in the manual. Darnit, I gotta just sit down and read that thing.

Section 6 API Operations goes into detail about how life has now changed and the free ride is over. It says "API operation requires that communication with the module be done through a structured interface (data is communicated in frames in a defined order)". Oh Joy.

I started hacking my Python app to see what kinda data I was getting myself into. The frame starts with a start byte of 0x7E and then followed by a two byte length. The IO data that I'm trying to get to is locked away in the Frame Data block that doesn't seem too bad.

Luckily before I started to make a career out of writing a API Frame parsing mess, I went back to Google to see what the trail blazers have done for me. I found the xbee.py file that ladyada used for her tweet-a-watt. Too bad it's for Series 1 and not a lot of help. But the header notes the author as Amit Snyderman. Wonder if he has a Series 2 version. Woo Hoo, he's got some google code at http://code.google.com/p/python-xbee But wait. The page says: The XBee API code is incomplete, missing support for valid checksums and an implementation for working with the Series 2 API. Darn it. Another bad lead.

Well, I'm on the Python Xbee page, I might as well be nosey and poke around. Maybe I can use the Series 1 code as a starting point and hack it up enough to get my single channel ADC data. Let's look it the Subversion source. What's that under the trunk? An XBee 2.5 folder. Woo Hoo!

Grab the xbee_api.py file, save it to your project folder and then for a quick test, write a second file like this:
#!/usr/bin/env python

from xbee_api import *

# replace first param with your serial port
a=XbeeApi("/dev/tts/0",9600)

Save this in a file called coord.py, make it executable (chmod +x coord.py), and run it (./coord.py). Now when sampled IO data is received, this is printed. Do I hear a Woo Hoo?
onData() ->{'code': 146, 'data': {'asamples': [3, 255], 'dsamples': [[0, 0, 0, 0, 0, 0, 1, 1], [1, 1, 1, 1, 1, 1, 1, 1]], 'dmask': [[0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0]], 'mac': [0, 19, 162, 0, 64, 62, 33, 116], 'samples': [1], 'address': [60, 237], 'amask': [[0, 0, 0, 0, 0, 0, 1, 1]], 'options': [1]}}
data {'asamples': [3, 255], 'dsamples': [[0, 0, 0, 0, 0, 0, 1, 1], [1, 1, 1, 1, 1, 1, 1, 1]], 'dmask': [[0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0]], 'mac': [0, 19, 162, 0, 64, 62, 33, 116], 'samples': [1], 'address': [60, 237], 'amask': [[0, 0, 0, 0, 0, 0, 1, 1]], 'options': [1]}
Now I need to figure out to interpret the data 'asamples'. But this is looking very promising. FYI, I'm going down this path to hook a thermistor to an XBee for my pool thermometer. More details to follow as I make more progress.

More FYI. I'm running this with OpenWRT on a wl-520gu hacked for my tweet-a-w/e. The receiving XBee is named BLUE. The second XBee is name WHITE and is connected to my laptop with an adafruit XBee Adapter that has ADC pin 2 setup for 20 second periodic IO sampling.

I've saved the today's version of my X-CTU config files here:
BLUE with ZNET 2.5 COORDINATOR API

WHITE with ZNET 2.5 ROUTER / END DEVICE AT with ADC pin 1 enabled and 20 second IO sampling.

What do you think? Leave a comment.

XBee Series 2 - JV Channel Verification

Damn it. Why do the simplest things always take hours to complete? I'm using XBee Series 2 cause I really didn't know what I was buying. Series 1 or Series 2? To a newb, sounds like 2 is the shiny. Why buy an out-dated series 1? Nobody wants last year's model.

If I would've done my homework, the series 1 (also called 802.15.4) are the most reliable, easiest to use. So of course, I'm using Series 2.

I first attempted to use them as described in the ZNet 2.5 Modules Product Manual, one as a coordinator, one as an end device. No luck. I followed every instruction I could find on the web. Nada.

Just by dumb luck, I found that if I flashed two radios as ZNET 2.5 ROUTER / END DEVICE AT and then set the Destination Address of each to point to the other, it appeared to work. I could send data to and from the serial ports. I used them in this way for the tweet-a-w/e foundation and had good success.

For an end device called WHITE that will talk to another end device named PINK, use X-CTU and load the ZNET 2.5 Router / End Device AT firmware. Go to the X-CTU Terminal tab and type:
+++
ATRE
ATNIWHITE
ATDNPINK
ATWR
Here's the X-CTU file for WHITE.

But now I'm trying to send analog data from one to the other and I'm back to nada. The data receive light blinks and it appears to read data, but nada bit comes out of the UART.

I decided to go back to square one and reconfigure as coordinator and end device. Sounds easy? Right. Took me two nights to get the devices associated. What's associated? From page 14 of the 802.15.4 manual (and not in the series 2 manual):
Association is the establishment of membership between End Devices and a Coordinator. The establishment of membership is useful in scenarios that require a central unit (Coordinator) to relay messages to or gather data from several remote units (End Devices), assign channels or assign PAN ID.
The series 2 manual seems pretty decent. I've read many sections and need to sit down and attempt to read more. Too bad though that it doesn't mention parameter JV Channel Verification. Which is the parameter that finally made a difference.
[update] maybe I was looking at the wrong manual. The one I ref'd above is the Series 2 Manual. The JV parameter is mentioned in the ZNET 2.5 manual.

When setting up a coordinator and associated end devices, there's a few things that you need to do:
  • After writing the firmware, go to the terminal and do a ATRE to reset to factory values. I would've saved lots of time if I realized that previous values were not being overwritten between firmware loads.
  • Make sure that the PAN ID is set the same on both devices. I suggest that while debugging, you have both devices connected to your computer via an adafruit XBee Adapter or SparkFun XBee Explorer and use X-CTU to monitor. You need to verify that the Operating PAN IDs are the same when they're trying to associate.
  • Set each device to a unique Node ID
  • The coordinator destination address should be set to DH=0, DL=FFFF
  • The end device destination address should be set to DH=0, DL=0
And for basic communications, that should be it. No need to set destination address of the end device explicitly to the coordinator. The 0's do that for you.

My problem though was my coordinator was on channel F and my end device was channel D. I could see this while watching them start up, but I could not figure out WTF.

Finally by going thru every parameter on both devices using X-CTU, I realized that JV Channel Verification was the ticket. This is described in the X-CTU tool as:
Set/read the channel verification setting. If enabled, a router will verify a coordinator exists on the same channel after joining or power cycling to ensure it is operating on a valid channel, and will leave if a coordinator cannot be found (if NJ=0xFF). If disabled, the router will remain on the same channel through power cycles.
In English, this means that if your devices are on different channels and JV is disabled, you're fricked cause the router won't go to another channel to associate.

Change JV to enabled and reboot the device:

+++
ATJV1
ATWR
ATFR

I've uploaded the configuration to my coordinator named BLUE that has ZNET 2.5 COORDINATOR API firmware. And an end device named PINK that has ZNET 2.5 ROUTER / END DEVICE AT firmware.

What do you think? Leave a comment.