Stock exchange alerts via instant message

Bulls and Bears

By Michael Schilli

The Pidgin instant messaging client [1] not only runs on a variety of operating systems, it supports a large number of IM protocols. Whether you prefer Yahoo or MSN, Google Talk or IRC, the free Pidgin plugin groups them under its umbrella, facilitating communications with online buddies who use different services. Pidgin's plugin architecture helps the omnipresent product keep on top of newly released or changing protocols and continually extends its scope.

You don't even need to write plugins in Pidgin's native language, C, and provide them as shared libraries. Scripts written in high-level languages like Perl can be linked in at startup, and if they cooperate with Pidgin's GLib-based event loop, they can even execute complex operations like retrieving websites while the Pidgin core and its graphical interface keep running uninterrupted.

Although it might sound crazy to let a C program play hopscotch with a Perl script in the same event loop, if you recall that Perl is just an abstraction layer on top of a C layer, and thus just as good at triggering and receiving events, the whole thing looks far more logical.

Ups and Downs

The Pidgin plugin that I will describe here is unobtrusive for the most part, but it regularly contacts a stock ticker service in the background. Because of this, it can let you know whether one of the stocks stored in the configuration file starts to climb like crazy or plummet. If either of these happens, the plugin opens a communication window to the logged in user, as shown in Figure 1, and displays the complete list of monitored stocks, including the day's changes as a percentage.

Figure 1: The stock exchange monitor alerting the user because the price of Google shares has risen by more than two percent.

In the configuration file shown in Figure 2, the user can specify which ticker symbols to monitor, as well as the scale of change that warrants an alert. For lines that contain the stock shortcut but no percentage, the plugin automatically assumes a tolerance of two percent.

The actual plugin code contains just 82 lines (Listing 1) and is fairly simple [2].

Figure 2: Users can add stock symbols to the watch list in the YAML file and define a threshold as a percentage if needed.

01 #!/usr/local/bin/perl -w
02 use strict;
03
04 use Pidgin;
05 use local::lib;
06 use Glib;
07 use WatchQuotes;
08
09 our %PLUGIN_INFO = (
10  perl_api_version => 2,
11  name => "Pidgin Stockwatch",
12  summary =>
13    "Stock Alert via IM",
14  version => "1.0",
15  author  => "Mike Schilli " .
16    "<m\@perlmeister.com>",
17  load   => "plugin_load",
18  unload => "plugin_unload",
19 );
20
21 our $USER = "yahoo-username";
22 our $PROTOCOL = "prpl-yahoo";
23
24 our $WATCH_QUOTES =
25   WatchQuotes->new();
26
27 #############################
28 sub plugin_init {
29 #############################
30  return %PLUGIN_INFO;
31 }
32
33 #############################
34 sub plugin_unload {
35 #############################
36  my ($plugin) = @_;
37
38  Purple::Debug::info(
39   "stockwatch",
40   "Plugin unloaded.\n");
41
42  1;
43 }
44
45 #############################
46 sub plugin_load {
47 #############################
48  my ($plugin) = @_;
49
50  Purple::Debug::info(
51   "stockwatch",
52   "Plugin loaded.\n");
53
54  $WATCH_QUOTES->init();
55  $WATCH_QUOTES->watch(
56   \&quotes_update);
57 }
58
59 #############################
60 sub quotes_update {
61 #############################
62  my ($msg) = @_;
63
64  Purple::Debug::info(
65   "stockwatch",
66   "Updating Quotes.\n");
67
68  my $account =
69    Purple::Accounts::find(
70   $USER, $PROTOCOL);
71
72  my $conv =
73    Purple::Conversation->new(
74   1, $account, $USER);
75
76  # user not online?
77  return unless defined $conv;
78
79  $conv->get_im_data->write(
80   $PLUGIN_INFO{name}, $msg,
81   0, time);
82 }

Extending Pidgin in Perl

To begin, Pidgin loads the code in Listing 1, assuming you've installed the Perl script with a .pl suffix in its plugin directory and assuming the user has activated the plugin in the corresponding dialog box (see the "Installation" section).

The plugin must define the callbacks required by the Pidgin API, plugin_init() and plugin_load(). Pidgin accesses plugin_init() to retrieve information about the plugin and be able to display it in the Tools | Plugins menu.

When stock market players click to enable the plugin later, the plugin_load function (lines 46-57) starts running. It defines the plugin's tasks by submitting them to the event loop, which will process them later. If the news from the market is so bad that the budding broker disables the plugin, it will handle any necessary cleanup in the plugin_unload() callback before Pidgin releases it from memory.

Outsourcing Quote Watch

The WatchQuotes module described later helps track stock quotes. It parses the plugin's configuration file when line 54 of the plugin calls its init() method. Line 55 then launches the permanent monitoring process by calling the watch() method. The plugin script passes a reference to the quotes_update callback defined in lines 60-82 to the method; the callback later wraps any message passed to it into an IM conversation and forwards the whole thing to the pleasantly, or unpleasantly, surprised user.

For this to happen, the find() method of the Purple::Accounts object in line 69 finds the user, who is (hopefully) logged in; line 73 then initiates a conversation to the user in the form of a Purple::Conversation object. If something goes wrong, say, because the user defined in $USER is not logged in to the specified service, line 77 returns, and the message never gets to see the light of a turbulent day at the stock exchange.

If everything works correctly, the write() method called in line 79 sends the message using the plugin name as the sender. The final parameter uses the time() function to set the current time, which Pidgin displays for each message.

Cool Kids' Framework

The $USER and $PROTOCOL variables in lines 21 and 22 define the username to alert in case of trouble, and the IM service in which to do this. Here, the service is prpl-yahoo, which is Yahoo's messenger protocol. The plugin script pidgin-stockwatch.pl pulls in the WatchQuotes module in line 7. When implementing this module, it's important to make sure that it processes events asynchronously with Pidgin's event loop. This ensures that the GLib kernel processes mouse clicks on Pidgins's UI by the user during long-winded operations, such as retrieving data from a website. If the plugin just issued web requests and waited for the data to trickle in, the GUI would turn into a pillar of salt until the result came back.

Several frameworks can be employed to accomplish this. I have used POE [3] in several Perl columns before, so this time, I'll give a newcomer a chance. The one that is the buzz with the cool kids of the Perl scene right now is the AnyEvent Framework [4]. It doesn't bind directly to a specific event loop but cooperates with half a dozen implementations.

Jack of All Trades

The advantage with using AnyEvent is that you can implement a module like WatchQuotes.pm to be totally generic and run it with all kinds of event loops and on a variety of platforms. The only precondition for this is that the main program maintains a reference to the AnyEvent object. If the last reference to it vanishes, AnyEvent will clean up after itself and no more events will be processed. To keep it running, I will store a reference to the WatchQuotes object in the global $WATCH_QUOTES variable, which then references AnyEvent's objects and keeps them alive.

This approach is necessary because Pidgin only calls the defined plugin callbacks briefly at startup, and their local variables disappear immediately after the program flow returns to the regular Pidgin biotope.

Bind Loosely

Loose event loop binding with AnyEvent offers another benefit: The WatchQuotes.pm module can be tested independently, of Pidgin, for example, by running the script in Listing 2. The script instantiates the WatchQuotes class and calls its init() method, which tells WatchQuotes to parse the ~/.pidgin-stockwatch.yml configuration file defined by the user and convert it internally into a Perl data structure (Figure 3).

Figure 3: Perl creates a data structure from the YAML file.

01 #!/usr/bin/perl -w
02 use strict;
03 use lib local::lib;
04 use AnyEvent;
05 use WatchQuotes;
06
07 my $watcher =
08   WatchQuotes->new();
09 $watcher->init();
10 $watcher->watch(\&callback);
11
12 my $quit_program =
13   AnyEvent->condvar;
14 $quit_program->recv;
15
16 #############################
17 sub callback {
18 #############################
19  print "$_[0]\n";
20 }

As a functional example of an event loop, AnyEvent defaults to a loop implemented in Perl. Listing 2 uses condvar() to define a conditional variable to which the AnyEvent kernel can send messages. The subsequent method call to recv() in line 14 then waits for a message, which never comes; in the mean time, it lets the AnyEvent kernel handle events from modules like WatchQuotes.

The test script used here retrieves stock prices at regular intervals and calls the callback defined in lines 17-20 with a list of formatted stock quotes, if the price of one of the monitored stocks defined in the configuration file exceeds the defined thresholds. The call to print() in line 19 then sends the message to stdout. The script continues to run and keeps refreshing stock data from the Yahoo ticker service until the user stops it by pressing Ctrl+C. This gives programmers a convenient approach to identifying any errors before the script is injected into the inimical Pidgin environment, where debugging is difficult - especially if Pidgin fails to load the plugin correctly for some reason (see the "RTFM - If You Have One" box).

The WatchQuotes.pm module in Listing 3 starts in typical Perl style by defining a new() constructor that creates an object and locates the user's home directory. It accepts additional parameters such as the name of the configuration file in conf_file but falls back to default settings if they're not provided and stores them in the object hash.

001 #############################
002 package WatchQuotes;
003 # Mike Schilli, 20100
004 # (m@perlmeister.com)
005 #############################
006 use strict;
007 use warnings;
008 use AnyEvent;
009 use AnyEvent::HTTP;
010 use YAML qw(LoadFile);
011
012 #############################
013 sub new {
014 #############################
015  my ($class, %options) = @_;
016
017  my ($home) = glob "~";
018
019  my $self = {
020   watcher => undef,
021   data    => {},
022   refdata => {},
023   conf_file => ("$home/" .
024     "pidgin-stockwatch.yml"),
025   conf => {},
026   %options,
027  };
028
029  bless $self, $class;
030 }
031
032 #############################
033 sub init {
034 #############################
035  my ($self) = @_;
036
037  my $yml = LoadFile(
038   $self->{conf_file});
039
040  for my $e (@$yml) {
041   if (ref $e eq "HASH") {
042    my ($key, $val) = %$e;
043    $val =~ s/%//g;
044    $self->{conf}->{$key} =
045      $val;
046   } else {
047
048    # 2% by default
049    $self->{conf}->{$e} = 2;
050   }
051  }
052 }
053
054 #############################
055 sub watch {
056 #############################
057  my ($self, $cb) = @_;
058
059  $self->{watcher} =
060    AnyEvent->timer(
061   after    => 10,
062   interval => 300,
063   cb       => sub {
064    $self->fetch($cb);
065   },
066    );
067 }
068
069 #############################
070 sub fetch {
071 #############################
072  my ($self, $cb) = @_;
073
074  my $url =
075      "http://"
076    . "download.finance.yahoo.com/d/"
077    . "quotes.csvr?e=.csv"
078    . "&f=spl1p2&s="
079    . join('+',
080       sort
081       keys %{ $self->{conf} }
082      );
083
084  http_get(
085   $url,
086   sub {
087    $self->parse_csv($_[0]);
088    $self->check($cb);
089   }
090  );
091 }
092
093 #############################
094 sub parse_csv {
095 #############################
096  my ($self, $csv) = @_;
097
098  for my $line (split /\n/,
099   $csv)
100  {
101
102   my (
103    $symbol, $prev,
104    $last,   $change
105     )
106     =
107     map { s/[^\w\.-]//g; $_ }
108     split /,/, $line;
109
110   next
111     unless defined $symbol;
112
113   $symbol = lc $symbol;
114
115   $self->{data}->{$symbol} =
116     [ $prev, $last,
117    $change ];
118  }
119 }
120
121 #############################
122 sub check {
123 #############################
124  my ($self, $cb) = @_;
125
126  if (!scalar
127   keys %{ $self->{refdata} })
128  {
129   $self->{refdata} =
130     { %{ $self->{data} } };
131  }
132
133  for my $stock (
134   keys %{ $self->{data} })
135  {
136   if (
137    $self->noteworthy($stock))
138   {
139    $self->{refdata} =
140      { %{ $self->{data} } };
141
142    # reset 'prev'
143    for my $s (
144     keys
145     %{ $self->{refdata} })
146    {
147     $self->{refdata}->{$s}
148       ->[0] =
149       $self->{data}->{$s}
150       ->[1];
151    }
152
153    $cb->($self->message);
154    last;
155   }
156  }
157 }
158
159 #############################
160 sub message {
161 #############################
162  my ($self) = @_;
163
164  my $msg = "\n";
165
166  for my $stock (
167   keys %{ $self->{data} })
168  {
169   my ($prev, $last, $change)
170     = @{ $self->{data}
171      ->{$stock} };
172   $msg .=
173 "$stock: $last $change%\n";
174  }
175
176  return $msg;
177 }
178
179 #############################
180 sub noteworthy {
181 #############################
182  my ($self, $stock) = @_;
183
184  my $price_ref =
185    $self->{refdata}->{$stock}
186    ->[0];
187
188  my $price_now =
189    $self->{data}->{$stock}
190    ->[1];
191
192  my $change_percent = abs(
193   (
194    $price_now - $price_ref
195   )
196  ) / $price_ref * 100;
197
198  return ($change_percent >
199     $self->{conf}->{$stock});
200 }
201
202 1;

YAML for Humans and Machines

The init() method defined in lines 33-52 calls the YAML module's LoadFile() function to parse the configuration. This format has the advantage of being just as easily readable for humans as for machines (Figure 2).

The application supports both simple array entries, such as - amzn, and hash data structures, such as - goog: 2%, which YAML stores as references to a hash providing the mapping ("goog" => "2%"). Line 41 checks to see if Perl's ref function returns the HASH string, which indicates a hash reference, to distinguish it from simple entries.

If an empty string is returned, the value is a simple scalar and line 49 inserts the default threshold value of two percent. The module stores monitored ticker symbols in the conf entry in the object hash and assigns the configured trigger percentage values to them.

The watch method in lines 55-67 creates a periodic timer. The after => 10 parameter tells the timer to call the callback defined in cb exactly 10 seconds after starting. This delay is intentional and necessary if the plugin starts before the user is reachable on the instant messaging network. Without this pause for thought, early messages would disappear into a black hole instead of going to the stockholder. The interval parameter, on the other hand, carries a value of 300 to tell the timer to call the callback every five minutes after the first run to pick up the latest stock prices from the Yahoo server and figure out if the current trades exceed the configured trigger values.

Asynchronous Web Fetch

The callback function reference passed to the watch() method points to the main script's quotes_update() function. The timer then passes it on to the fetch() method (lines 70-91), which retrieves the ticker data from Yahoo's website.

Although this can take a couple of seconds if the going is tough on the Internet, the http_get() function called in line 84 comes from the treasure trove of the CPAN AnyEvent::HTTP module and processes the request asynchronously. The function expects the URL for the stock price service and a callback, to which it jumps once the data has trickled in completely.

Note that Perl immediately carries on with the program flow after calling http_get(), without the requested HTTP data being available at this point in time.

Free Ticker Data

As the documentation for the CPAN Finance::YahooQuote [9] module reveals, Yahoo's stock ticker service supports a whole bunch of parameters, from which the WatchQuotes.pm picks only those listed in Table 1: the ticker symbol (s), the previous day's price (p), the current price (l1), and the percent change (p2).

Bundled together as a string, WatchQuotes throws spl1p2 plus s=goog+yahoo at the Yahoo service for my sample user, who is interested in how the Google and Yahoo stocks are faring.

The data are returned in CSV format, which is two lines of text like the following:

"GOOG",467.49,475.83,"+1.78%"
"YHOO",14.89,14.94,"+0.34%"

The simple regular expression-based parser in parse_csv() (lines 94-119) creates a data structure from them. The data hash entry of the WatchQuotes object then contains a reference to an array that contains the previous day's price, the latest price (typically with a delay of 20 minutes), and the change (%).

To Alert or Not To Alert

It remains to be seen whether the price fluctuations warrant alerting the user; this is handled by the check method in lines 122-157. Again, the callback that will potentially use IM to contact the user is passed to the method.

The code in line 129 copies the latest set of data from the {data} object entry into the {refdata} section of the archive to give the code a reference for comparison purposes later. Instead of just saving the reference itself, the { %{ $self->{data} } } line copies the data held by the $self->{data} reference, creates a new hash from the results, and returns a reference to it.

If archived data are available from previous calls, the for loop in lines 143-145 would modify the entries for the previous day's price to match the current price. I don't want the code to send a new message every five minutes for a stock price that has risen once, but I do want to know if the price continues to change and again exceeds the threshold.

noteworthy() (lines 180-200) checks to see whether the stock price has exceeded the previous day's price by configured margin (or simply the previous price if an alert has already occurred today). In this case, line 153 calls message() in line 160 to format the data of all monitored stocks and to create a text string. The same line then calls the passed-in Pidgin plugin callback function, which uses IM to alert the user.

Installation

Popular Linux distributions will offer a package for the Perl interface to Pidgin (e.g., Ubuntu: pidgin and libpurple0, both version 2.7.0). The pidgin-stockwatch.pl plugin script (Listing 1) needs to be made executable and stored in ~/.purple/plugins below the budding broker's home directory. Note that the .pl suffix is required, or Pidgin won't pick it up. The WatchQuotes.pm module (Listing 3) must be in a path where the local Perl installation can find it. If necessary, the script code will point to the right location, as you can see from the use local::lib instruction in line 5 of Listing 1. What's local::lib, you might ask?

The AnyEvent and AnyEvent::HTTP modules haven't made their way into some popular distributions - and you might need to visit CPAN to retrieve them. To avoid messing up the Linux package manager's clean Perl packages with additional CPAN modules, users who appreciate a tidy system will use the local::lib CPAN module to install them below ~/perl5 in their home directories. After entering:

perl Makefile.PL --bootstrap
make test && make install

in the downloaded local::lib distribution from CPAN, you will probably want to append the output of the command

perl -I$HOME/perl5/lib/perl5 -Mlocal::lib

to your local .bashrc file. After restarting the shell (or sourcing the .bashrc file), local::lib sets environmental variables that tell the CPAN shell to install various modules in ~/perl5 under the user's home directory and to point Perl scripts you call to the additional search paths. But, Pidgin remains unaware of this and needs an explicit use local::lib instruction to put it on the right track.

Hurdles for Developers

The plugin script in Listing 1 will not run without Pidgin and throws irate error messages at anybody bold enough to try. The messages hint that it is unable to find some GLib functions in the corresponding shared libraries.

According to Pidgin developers, this is normal, although your opinion might differ. To get it working as a standalone script for testing, you can use a temporary workaround and just comment out the use Pidgin and use Glib lines. At least this way, you'll be able to type the line

perl -c pidgin-stockwatch.pl

and check that the syntax is okay and that the script finds the WatchQuotes.pm module and the CPAN modules it needs. If you launch Pidgin in debug mode by entering pidgin -d, it will provide detailed output on what is currently going on and you can even add more messages from within the Perl plugin by using:

Purple::Debug::info("stockwatch", "Plugin loaded.\n");

At least that way you can see what the plugin is doing and confirm that Pidgin finds it. If this works, and if Pidgin at least runs the plugin_init() routine in lines 28-31 of Listing 1, the plugin will appear as the name specified in line 11, Pidgin Stockwatch, in Pidgin's Tools | Plugins menu (Figure 4). Clicking the checkbox on the left enables the plugin and runs its plugin_load() routine. For test purposes, you can disable the plugin again, after which Pidgin will call the plugin_unload() routine in line 34.

Figure 4: A mouse click enables the newly installed plugin, which Pidgin lists in the Plugins menu.

After doing all of this, the budding broker can then populate the ~/.pidgin-stockwatch.yml configuration file with ticker symbols for the stocks to be monitored, assign percentage values, or accept the default of two percent. After restarting, Pidgin works with the updated values.

On a quiet day at the stock exchange, you will probably forget that the plugin has been installed at all. But as soon as the stocks go on a roller coaster, you'll be alerted and can quickly contact your favorite broker and engage in panic buying or selling.