Building Enterprise

Applications with Mozilla, AMP,

XML-RPC and JSON
● Jay Sheth
● http://www.moztips.com
● Presentation for NYPHP Group
● IBM, 590 Madison Ave
● New York, NY
● Tuesday, May 24, 2005
 Ajax, Ajax, Ajax!
What is a Firefox + LAMP application?
It's an Ajax-like application
AJAX: Asynchronous JavaScript + XML
http://www.adaptivepath.com/publications/essays/archives/000385.php

Client-Side:

* XUL
* JavaScript

Server Side:
* PHP
* MySQL

Data Transport:
* XML-RPC
* JSON

AJAX-like technologies are being used by companies such as Google in Google

Suggest (http://www.google.com/webhp?complete=1&hl=en ) and
Google Maps ( http://maps.google.com/ ).
A Bit of Architecture
 It's All About the
Central Business Logic Hub
All business logic is centralized in the form of a central PHP-driven hub
●

Both administrators and end-user can access the same business hub
●

Administrators use a rich web client, delivered on-demand to Firefox

●

End-users use any generic HTML-based browser

●

Thus the same data can be accessed and modified

by different web clients that access a single central engine.

Thus code duplication between administrator and

end-user functionality is avoided.

End-user website can be displayed using PHP 4, PHP 5

or even (heaven forbid) ASP.net.
 DB Abstraction at the Backend Business Hub

Putting PEAR DB to use enables easy database portability

 Enough Talk – Let's See an Example
Central Business Logic Hub as Implemented at
www.brooklynandbeyond.com
End-user site: directory of restaurants, real estate and other businesses in Brooklyn, Manhattan, Queens, NJ and CT

Administrator's View:
Example, Part II
End-user site
 Recap: The Application

* Database of restaurants and other businesses in Brooklyn, Manhattan,

New Jersey and Connecticut

* Two views:
Rich Client for Admin (XUL + JS + PHP + MySQL)
– Status: Beta

Flat Website for end-user (HTML + CSS + PHP + MySQL)

– Status: Alpha / Proof of Concept
 What's so good about
a rich web client ?
● Web App + Desktop Application Hybrid
● Real time response reflected in user interface
● Provides 'rich', instead of 'flat' experience
● Leverages Mozilla platform for improved user
interface widgets (such as editable combo
boxes)
● Tabbed display paradigm enables easy
editing of large number of data fields
 Recipe Ingredients
Making a nice cup of XUL, JavaScript, PHP and MySQL Noodles

* Start with some PEAR libraries as the base - http://pear.php.net

HTTP Request
Net Socket
Net URL
XML-RPC
DB

* Add in Jsolait JavaScript libraries for good measure - http://jsolait.net/

*Add in PHP JSON server library - http://mike.teczno.com/json.html

* Add some PHP code to make an XML-RPC server (stir well)

* Add some PHP code to make a yummy index.php file
* Add some (well a lot of, actually) XUL and JavaScript code
to the index.php file so that it can 'talk to' the XML-RPC server

* Open the 'content' packet, and pour it into the MySQL DB.

You have one hot cup of Mozilla LAMP noodles there!

 Show us some code already !
A basic XML-RPC transaction
HTTP Request:

POST /admintwo/mailingdbserver.php HTTP/1.1

<?xml version="1.0"?>
<methodCall>
<methodName>system.listMethods</methodName>
</methodCall>

HTTP Response:

<?xml version="1.0" encoding="UTF-8"?>

<methodResponse>
<params>
<param>
<value><array>
<data>
<value><string>getAllEntriesBrief</string></value>
<value><string>getAllCategories</string></value>
<value><string>getTelLookupData</string></value>
</data>
</array></value>
</param>
</params>
</methodResponse>
Cool Feature: Auto-address lookup
using reverse phone lookup
Step 1) Enter a phone number
Auto-address lookup (II)
Step 2) Press the reverse phone lookup button
 Code for Auto-address Lookup (I)
XML-RPC Server-Side Code
require 'google_lookup.php';
/*
Use Google to do a reverse lookup of a telephone number, to get the contact information associated with that number.
Return that information.
*/
function get_tel_lookup_data($params)
{

$tel_no = XML_RPC_decode( $params->getParam(0) );

// Check the telephone number for valid syntax before performing a lookup
$tel_parts = extract_tel_digits($tel_no);

if ( ! is_array($tel_parts) )
{
return new XML_RPC_Response(0, $XML_RPC_erruser + 1, "Error: phone number invalid.");
}

$my_lookup = lookup_google('http://www.google.com/search', $tel_no);

$retval = XML_RPC_encode($my_lookup);
return new XML_RPC_Response($retval);
}
 Code for Auto-address Lookup (II)
Google lookup function (excerpt)
require_once 'HTTP/Request.php'; // PEAR HTTP_Request Library to POST data (or to GET data)

function lookup_google($remote_server, $telephone)

{
$req =& new HTTP_Request($remote_server);
$req->setMethod(HTTP_REQUEST_METHOD_GET);
$req->addQueryString('q', $telephone);

if (!PEAR::isError($req->sendRequest())) { $response2 = $req->getResponseBody(); }

else { $response2 = 0; }

if (! empty($response2) )
{
$num_lookups = 0;
$our_matches_all = array();
$response_no_tags = strip_tags($response2);
$response_no_tags = str_replace(' & ', ' and ', $response_no_tags);

// Make space after Mapquest links so that after HTML tags have been stripped, there is a space
// before the business name

$response_no_tags = str_replace('MapQuest</a>', ' ', $response_no_tags);

$num_matches = preg_match_all('/([A-Za-z\s-]+),\s\((\d{3})\)\s(\d{3})-(\d{4}),\s([0-9-]+ [\w\s.]+[.#0-9\s]*),\s([A-Za-z\s]+),\s([A-Za-z]{2})\s(\d{5})/',

$response_no_tags, $our_matches_all, PREG_PATTERN_ORDER);

/*
More stuff happens here ...
*/
// Populate and return new array with relevant information from $our_matches_all
}
}
 Code for Auto-address Lookup (III)
XUL widgets population / JavaScript code
function getTelLookupData(tel_no)
{
set_waitbox('-- please wait --');
var methods = [];
try
{
var server = new xmlrpc.ServerProxy(globalXmlRpcServer, methods);
var lookup_data = server.getTelLookupData(tel_no); // calling remote XML_RPC function here!
// Automatic type conversion here from PHP assoc array to JS object
if ( lookup_data['rpc_msg'] != 'No reverse lookup matches found.' && lookup_data['rpc_msg'] !=
'Reverse Lookup: an error has occured.' )
{
document.getElementById('b_name').value = lookup_data['b_name'];
document.getElementById('b_phone').value = lookup_data['b_phone'];
last_phone_num_lookup = lookup_data['b_phone'];
document.getElementById('b_street_address').value = lookup_data['b_street_address'];
document.getElementById('b_city').value = lookup_data['b_city'];
document.getElementById('b_state').value = lookup_data['b_state'];
document.getElementById('b_zip').value = lookup_data['b_zip'];
}
else
{
// Turn off auto lookup if no phone number has been found, or if an error has occured
toggleReverseLookup();
toggleALButton();
}
set_waitbox( lookup_data['rpc_msg'] );
}
catch(e)
{
set_waitbox('Error:' . e);
alert(e);
}
}
 Automagical zip population

Similarly, when an entry is made and both zip codes are left blank, they are
looked up.

If an appropriate match could be found, this information is saved

to the database.

Otherwise, an error is exposed as a JavaScript alert, and a color and text

change in the notice box. (More on the notice box in a later slide.)
 Error Checking
* Centralized on the server side
* Exposed on the client side as native JavaScript alerts / actions
Nice feature: editable drop-downs
XUL has editable drop-downs – a big improvement over HTML's select tag
 XUL Code for Editable Dropdowns

<hbox>
<label control="b_neighborhood" value="Neighborhood:"
style="width:100px;" />

<menulist editable="true" id="b_neighborhood">

<menupopup>

<menuitem label="--select--" value="--select--"/>

</menupopup>
</menulist>
</hbox>
Note: neighborhood values are dynamically added using the DOM via
JavaScript
JavaScript Code for Dynamic Neighborhood
Drop-down Population
//Get a list of distinct neighborhoods from the entries table

function getAllNeighborhoods()
{
var methods = [];
try
{
var server = new xmlrpc.ServerProxy(globalXmlRpcServer, methods);
var all_neighborhoods = server.getAllNeighborhoods(); // calling remote XML_RPC function here!

// Automatic type conversion here from PHP assoc array to JS object

var menuList = document.getElementById('b_neighborhood');

var menuList2 = document.getElementById('s_near_neighborhood');

for (var i in all_neighborhoods)

{
aneigh = all_neighborhoods[i];
neighborhood = aneigh['neighborhood'];

if (neighborhood)
{
menuList.appendItem(neighborhood, neighborhood);
menuList2.appendItem(neighborhood, neighborhood);
}

catch(e)
{
alert(e);
}
}
PHP Code for Dynamic Neighborhood
Drop-down Population
//Get a list of all distinct neighborhoods in the entries table

function get_all_neighborhoods()
{
global $username, $password, $server, $database, $mailing_table;
global $XML_RPC_erruser;

$dbh = DB::connect("mysql://$username:$password@$server/$database");

if ( DB::isError($dbh) )
// CHECK PEAR DB CONNECTION ERRORS
{
// Return XML-RPC fault
return new XML_RPC_Response(0, $XML_RPC_erruser + 1, "DB Error: Could not connect to server.");
}
else
{
// Retrieve list of distinct neighborhoods
$dbh->setFetchMode(DB_FETCHMODE_ASSOC);

$query = "SELECT DISTINCT neighborhood FROM $mailing_table ORDER BY neighborhood ASC";

$mlist_neigh = $dbh->getAll($query);

if ( DB::isError($mlist_neigh) )
// CHECK FOR PEAR DB / QUERY RUN ERRORS
{
return new XML_RPC_Response(0, $XML_RPC_erruser + 2, "DB Error: Could not run query.");
}
else
{
$retval = XML_RPC_encode($mlist_neigh);
return new XML_RPC_Response($retval);
}
}
}
Nice feature: PEAR QF-like hier-select menus
Step 1: select category from editable-dropdown
Hierselect (Part II)
Step 2: select from list of subcategories pertaining to this category
 Code for XUL Hierselect Menus

Code Link :
http://www.moztips.com/wiki/index.pcgi?page=XuLHierMenulists

Live Demo:
http://www.moztips.com/code/mozbugs/menulist/menulist_bug.php
Capitalizing on MySQL's FULLTEXT indexing
for listings search
● MySQL is great at storing, searching through and filtering data

●Deliberate compromise in DB design: de-normalized table structure

●Category, Subcategory, Country Affiliation etc. stored in same table

●These fields stored as VARCHAR(255)

●Created a MySQL FULLTEXT index for these, and other fields

Sample search queries:

●Bronx Italian

●Bay Ridge pizza

●Japanese restaurants

●Vegetarian

●West Village Italian

Sample FULLTEXT query (simplified):

SELECT id, MATCH (title,body) AGAINST ('Tutorial') FROM articles;

More info.:
http://dev.mysql.com/doc/mysql/en/fulltext-search.html
MySQL FULLTEXT Search Example (I)
Content Administrator's View
MySQL FULLTEXT Search Example (II)
End User's View
 Problem with rich web applications and large datasets:
things can slow down, instead of speeding up

Things slow down because:

●Large amounts of separate pieces of data are loaded on each 'event' (e.g. editing an entry)
●Choosing XML-RPC as the underlying data transport is convenient but also expensive

●Parsing XML on the client-side is resource-intensive

How to speed things up:

●Optimize application design for speed
●Reduce the amount of requested data by using pagination

●Use an alternative data transport format, such as JSON (JavaScript Object Notation)
 Techniques for Optimizing Rich Web Applications for Speed
(On to-do list)
●Intelligently cache data locally in either JavaScript arrays or XML files
●Local XML file caching is more complicated, as Firefox requires local file access

●Ensure that JSolait library does not request list of remote methods on each web service request

●When data for one listing is retrieved, also preload data into JavaScript array for surrounding 10 listings

●Only reload data from remote DB when data on the client side is stale

Case in point: reloading complete listings list takes too long

 Using JSON for Optimized Data Transport Speed
JSON: JavaScript Object Notation

●JSON is a JavaScript Object literal

●JavaScript objects are the same as JavaScript associative arrays

Example:
{"myname":"Jay","place":"New York"}

PHP Equivalent:
array('myname' => 'Jay', 'place' => 'New York')
 How to Get Data via JSON

* Mozilla makes a GET request to page such as:

http://localhost/json/test_json.php?sort=az

* That PHP page queries MySQL

and lists all entries (such as weblog posts and ids)
in alphabetical order in JSON format ,
using the JSON PHP library

* Mozilla reads this object literal string from the PHP-generated page,
converts it into a JavaScript Object / Assocative Array,
using the eval() function

* Mozilla loops through the JavaScript Object,

and enters each title into a XUL Listbox

Using JSON results in a 600 - 700% speed increase over XML-

RPC.
Some JSON Request Code (I)

init_urllib.js

// init_urllib.js
var urllib=null;
try
{
var urllib = importModule("urllib");
}
catch(e)
{
reportException(e);
throw "importing of urllib module failed.";
}
 Some JSON Request Code (II)
HTML / PHP Page:
<!--
This page is:
http://localhost/json/test_json.html /
http://localhost/json/test_json.php
-->
<script type="application/x-javascript" src="jsolait/init.js"> </script>
<script type="application/x-javascript" src="jsolait/init_urllib.js"> </script>
<script type="application/x-javascript" src="jsolait/lib/urllib.js"> </script>

<script type="application/x-javascript">
var rslt = urllib.sendRequest("get", "http://localhost/json/test_json.php");

if (rslt.status == '200')
{
jsonSerialized = rslt.responseText;

//eval( 'obj3 ={"myname":"Jay","place":"New York"};' );

eval( jsonSerialized );
document.write(obj3['myname']);
document.write(obj3['place']);
}
else
{
alert('Error: the server could not find the specified file.');
}
</script>
 Some JSON Request Code (III)
PHP Script (loaded via GET HTTP Request)

<?php
// test_json.php

require_once('JSON.php');
$json = new JSON();

// convert a PHP value to JSON notation, and send it to the browser

$value = array('myname' => 'Jay', 'place' => 'New York');

// $value is:
// {"myname":"Jay","place":"New York"}
$output = $json->encode($value);
$output = 'obj3 =' . $output . ';';
print($output);
?>
 Powering the End-User Site with PEAR XML-RPC Client
and PEAR Sigma Templating System (I)

(bonus material)
 Powering the End-User Site with PEAR XML-RPC Client
and PEAR Sigma Templating System (II)
(bonus material)

PHP Fulltext Search Code Excerpt (some code omitted!)

<?php
// Use MySQL's fulltext searching to search DB
$s = $_REQUEST['s'];
// Include preference file, PEAR XML-RPC Client, and PEAR Sigma Template Library
// Instantiate Sigma template object. Set the templates directory to the subdirectory called 'templates'

if ( strlen($s) > 0 )
{
$client = new XML_RPC_Client($xmlrpcpath, $xmlrpcserver, 80);
$client->setCredentials($xmlrpcusername, $xmlrpcpassword);

$message = new XML_RPC_Message('getEntriesByKeyword

getEntriesByKeyword');
$message->addParam( new XML_RPC_Value($s, 'string') );
$response = $client->send($message);

$return_value = $response->value();
$search_results = XML_RPC_decode($return_value);

$num_s = count($search_results);
for ($j = 0; $j < $num_s; $j++)
{
$row = $search_results[$j];
$tpl->setVariable(
array(
'name' => $name,
'id' => $row['id'],
'street_address' => $row['street_address'],
'address2' => $row['address2'],
'cross_streets' => $row['cross_streets'],
'city' => $row['city'],
'state' => $row['state'],
'neighborhood' => $row['neighborhood']) );

$tpl->parse('sitem_block');
}
$tpl->show(); // Display the newly created HTML file
}
else {$tpl->show(); // Display the newly created HTML file}
?>
 Powering the End-User Site with PEAR XML-RPC Client
and PEAR Sigma Templating System (III)
(bonus material) S
Sigma Template Code Excerpt i
<html>
g
<head>
<title>Brooklyn and Beyond </title> m
</head>
<body>
<div class="mainbody">
a
<h3> Search Listings </h3>

<p>{num_results_msg}</p>

<p>{no_results_msg}</p> g
<p> {search_tip} </p> h
<!-- BEGIN sitem_block -->
<ul>
<li>
<strong><a href="detail.php?id={id}">{name}</a></strong>
<br />
{street_address}
<br />
{cross_streets}
<br />
{city}
<br />
{state}
<br />
{neighborhood}
</li>
</ul>
<!-- END sitem_block -->

</div>
</body>
</html>
 Conclusions

●AJAX-like applications can dramatically improve user experience

●XUL + JavaScript on the client is perfect for data entry tasks

●Having each component separated from the other makes for 'hot-swappable'

components, and an easy-to-upgrade site

Hot-swapping components – some examples:
If you (or the pointy-haired one) decide tomorrow that it would be way cool to use
ASP.net to display the end-user site, you could easily do so. Simply acquire an XML-
RPC client for ASP.net, and rebuild the end-user portion. The central business hub will
still be powered by PHP 4 or PHP 5, and the ASP.net site will access data via the same
XML-RPC API.

Suppose you would like to migrate from MySQL to PostgresQL for the datastore. Simply
import your data into PostgresQL, change the PEAR DB DSN strings in the central
business hub, and modify queries slightly to account for the differences between
MySQL and PostgresQL.

It's as easy as that! No, really!

Questions?
Comments?
Thank You!