Summer holidays have been wrung out to the last drop. Kids are back at school. Time to start on 2008.

RUBRIC is now officially finished, except that we are keeping the tech team running until the end of March to continue limited support for the RUBRIC partners. The RUBRIC tech team have joined me in the Learning Futures Innovation Institute; the LFII.

So it’s time to start thinking about what we’re going to do in out new institute. Of course we’re going to have to establish our plans through proper channels but I will talk a bit about the projects we’ve already got running and a few ideas for the future.

There are two groups that come under my area, in my new role:

1. The Software R&D lab, where they’re working on the Integrated Content Environment (ICE); integrating it with Learning Management Systems (LMSs) and institutional repositories (IRs).

2. The new USQ Repository Services group, led by Caroline Drury, where they’re doing the tail-end of RUBRIC tech support and looking after three clients (some of the contract details are still a bit up in the air so I won’t name names).

Immediate future

The Repository services group are fully booked until the end of March, with contracts continuing to the end of 2008. We need to see what role we are able to play in the new Australian National Data Service (The ANDS Technical Working Group 2007).

For the R&D group we’re working a major overhaul of the ICE interface and continuing to work on the server-based version of ICE.

During the last quarter the ICE system has been moving towards a service-oriented approach (that’s ’small soa’ as they use the term in the e-Framework for Education and Research). This means we have broken the previously monolithic ICE into smaller pieces, and tried to make parts of it available for use with other systems.

One example of a service is a forthcoming content conversion service for the Moodle LMS which will let you upload structured word processing documents, created using the ICE toolbar / templates into Moodle and return good quality HTML, unlike the usual garbage you get from saving as HTML in your word processor or using services like Google Docs.

Another example is being coded-up right now by Oliver Lucido. He’s working on a service to let ICE authors include visualizations of stuff encoded in Chemical Markup Language (CML) in their documents. I wrote about this mid last year with a simple demo, now we’re putting together a framework to support this kind of integration, which we hope to extend when Peter Murry-Rust visits us in a couple of weeks. CML, of course is not everyone’s cup of tea but. I’m hoping that our work on tools for chemists will expand into tools for other researchers in other disciplines. We will continue to work on general purpose stuff like authoring plugins for the Zotero bibliographic application.

The future

Both the groups I’m working with are cogs in a big university machine, so it’s not really up to us what we do, but here are a few things that I think we have a good chance of working on:

1. Most importantly, the Software group will be driven by USQ’s researchers in flexible learning, developing new tools on top of the IT platform we’re using at USQ. The platform includes Moodle, ICE and ALIVE (serious games), with some proprietary tools for collaboration.

   I don’t know what software we will need to write, that depends on what the research tells us to try.

2. Packaging the tools that we’ve developed and adapted at USQ for developing and delivering distance education, so that they can be used by others, particularly to produce Open Courseware.

3. Continuing the work we’re doing on scholarly authoring and its relationship to data, with tools for researchers and students.

4. And finally, if there’s time we might investigate some of the things I’m thinking about now. Here’s three that spring to mind, but there are lots more:

   • The One Laptop per Child Project (note to my kids: this does not mean you. I’m thinking three or four laptops for daddy and one for the rest of the family to share). We should at least look at getting a virtual one.

   • An online document conversion service using ICE technologies running on something like Amazon’s Elastic Compute Cloud, possibly generating money.

   • Collaborative online document editing applications. Stijn Dekeyser from USQ’s Maths and Computing is interested in collaborating on this one.

The ANDS Technical Working Group. 2007. Towards the Australian Data Commons. A proposal for an Australian National Data Service. Canberra: Australian Government Department of Science Education and Trainining http://www.pfc.org.au/twiki/pub/Main/Data/TowardstheAustralianDataCommons.pdf.

I have complained here over and over about how bad word processors are at producing HTML, at least until you see the light start using the ICE system to style and convert your documents. I documented some of the problems last year when I looked at how an author might fare trying to write a paper for the web using Microsoft Word, OpenOffice.org Writer or Google Docs.

But now I am delighted to report that IBM have solved this problem. The latest IBM Lotus Symphony does not have the usual problems in its HTML export. No garbled code. No messed up lists. No stupid attempts to reproduce paper formatting on the web.

Why?

‘Cos it has no HTML export at all.

Now I can “Be Free. Work Smart.”

OK, so this is a beta release and I was running it on Ubuntu Linux which is not supported. Maybe the final release will have spectacularly good HTML export.

I didn’t spend a lot of time with the word processor but I found it confusing. Apparently it’s based on an outdated version of OpenOffice.org Writer with the menus moved around and renamed, possibly to fit with the old Lotus way of doing things. Dig below the surface and you can see the old OpenOffice.org dialogs in there, though.

Peter Murray Rust followed up my post on publishing interactive maps, with a post about graphing.

XY graphs are so common and so important that we ought to be able to cut and paste them, preserving he semantics of the underlying data. One of my New Year WIBNIs. [Link added by ptsefton as I had to look up WIBNI]

http://wwmm.ch.cam.ac.uk/blogs/murrayrust/?p=905

As I said in the last post, what I’m trying to do, with the help of the ICE team is to find ways for people to publish both data and visualizations of the data, preferably interactively.

Why?

Well an example is; in my last post there was an anomaly in the data, or the graph of the data. I’m not sure which, but it should be possible for people to find out, because the data is all out there in the open. More on that later.

Inspired by Peter’s WIBNI, I’ve been poking around the web this morning to see what graphing tools are out there for HTML, and found the modestly titled Definitive guide to charts and graphs which surveys methods of embedding graphs in HTML.

I’m still thinking about the possibilities, but here are a few quick experiments with graphing, using OpenOffice.org Writer (NeoOffice) and the ICE system. As you will see, this post is far from a definitive guide.

Google Chart API

First up, I explored the Google Chart API. As it says on the site, The Google Chart API lets you dynamically generate charts using a URL. All the data are encoded in the URL itself, which makes copying and pasting really easy, but you would need to find the API document to decode the data.

Here’s an attempt to reproduce one of the diagrams we use to explain ICE. The Google Chart API will only let me have three circles so it doesn’t really work in this case.

Here’s another chart showing the average rainfall for Toowoomba in mm, using data that I downloaded from the Bureau of Meteorology in CSV Format. To use the data I converted it to a percentage of 150mm – hope I got it right.

Follow the link on the picture to see the data encoded in a URL– but there has been some data loss because of the requirement to encode the data for the Google Chart API.

Flot

Next I tried Flot (found via a comment on PMR’s post).

To make this work I used the same technique I wrote about for maps. I put a copy of the Flot HTML page on my server, took a screenshot, pasted it in here and linked it to the HTML. ICE used the HTML for online use and the image for print/PDF.

[Update: Fixed! WP has this bizarre feature that adds paragraph tags to your code when it renders a page, even in the middle of a script. I turned it off using a plugin called Disable WPAUTOP.]

Chart tool in Writer

Finally, I did the obvious thing and tried the OpenOffice.org Chart tool. This was by far the quickest and easiest way to go. I put the data into a table, clicked Insert, Object, Chart and I was away. It’s not interactive for the end user, but I can click on it to change the chart, as could you if I gave you access to my source document or exposed the chart as a download that you could play with in an OpenDocument aware application.

Raw data

And the winner is?

As far as I can see, word processing users are reasonably well served with tools for making static charts. The point-n-click wizards in Microsoft Office and OpenOffice.org are hard to beat for usability. The best thing to do now is probably to expose the data so that others can graph it themselves. There’s no way that most authors will want to use something like Flot or the Google Chart API until there are wizard tools to let them point at a table of data and graph it with a few clicks.

None of the solutions I looked at today that are close to meeting Peter’s requirement; XY graphs are so common and so important that we ought to be able to cut and paste them, preserving he semantics of the underlying data:

1. The Google Chart API format is cute – and it’s very copy-and-pastable, but it:

   1. Is very painful to use.

   2. Does not work with raw data – you have to encode the data, thus loosing its integrity.

2. Flot is also cute, but is also a pain to use, and copying and pasting is out of the question at present as the data are embedded in Javascript in an array.

3. The office-suite based solution is sub-optimal because it’s not a web format but it seems to be the best compromise at present.

For options 1 and 2 above it wouldn’t be too hard, though, to write some code to pull data from a table in your document, and slap on an easy to use front-end.

Of course this was a survey of ad hoc charting tools – there’s lots of scope to automate things where people are using the same kind of data over and over again. If, for example you wanted to deal with rainfall data a lot you could set up an automated process to take the data and turn into an Google Chart API URL or a Flot chart.

(I have left in one deliberate error I made with the data for keen readers to discover - I wonder how many accidental errors there are though?)

[Update 2008-01-03 – fixed a technical issue]

I showed my recent post on embedding maps into blog posts to my long-suffering partner.

She said that while the embedded Google map was cool, the explanation was incomprehensible to her. Me, I would prefer that even if some of the technical detail went over her head that she at least got the point, which apparently she didn’t, on account of it was, well, incomprehensible.

(And then she went on to read a few more posts and pointed out some really bad sentences that I had written in the recent past. I think it’s my eyes that are causing the problem; they don’t focus on text like they used to, but it could well be my brain.)

Here’s another try at the same post with the word processor zoomed in to 150% so I can see what I’m typing.

The end result: an interactive map

First up, I invite you to admire this map which shows the route of a bike ride I went on with the Toowoomba Bicycle Users Group.

If you click on the map you can drag it around.

Use the + and – buttons to zoom in and out.

And if you click on the title at the top (open it in a new window) it will take you to Bikely where you can look at the ups and downs of riding on the Darling Downs by clicking on Show then Elevation profile.

Which shows you this:

(Note: I think there’s something wrong; it shows a very dramatic dip in the ride that I don’t remember, around the 26km mark).

But what am I trying to achieve?

This is not just about putting up maps on a website. We are trying to develop tools so that authors can embed all kinds data into their publications and then visualize and explore and verify the data. As we work for a university we are particularly interested in this for educational and research purposes. We want to give authors tools to do three things:

1. Allow them to capture or link to some data.

2. Generate a static 2d representation of the data that they can paste into a word processing document and use for print or PDF.

3. Publish rich interactive views of their data to the web:

Instead of publishing a bike route as a picture, why not embed an interactive map in a web document?

Instead of publishing a chemical formula for a molecule, why not show an interactive 3d model?

Instead of citing chapter and verse in a classical text, why not have the citations turn into links to multiple translations of the text? (Have a look at this list of tools and think about how they could be integrated with an online text.)

Instead of a table of data or a static chart, why not show a chart tool where the reader can zoom in an out, and otherwise explore the data?

This stuff is not easy at the moment. For example see this post by Peter Murray Rust about the difficulties of trying to paste a fragment of Wikipedia code into his blog, following on from an earlier post. Peter mentions the Integrated Content Environment project that I lead at USQ where we are trying to help solve these problems.

Now, I’m going to explain how I captured the data to make this map, and describe some of the challenges involved, and finish up by looking at some of the issues raised.

How do you make a map like this?

If you want to make a map like mine, the short answer is: join Bikely and draw one yourself using the instructions on the site. It’s a simple-yet-fiddly process of clicking on a map to draw your route.

But there is a much more complicated way which, of course, is what I decided to do.

I decided to use a GPS device to record a bike ride, upload the data onto my computer, then upload that to Bikely. I used a free program called GPSBabel to do the upload. (I had trouble with the USB cable I bought for my Garmin eTrex GPS. If you’re planning to do this maybe choosing the absolute cheapest cable off eBay is not the best plan.)

Once I got the upload working I ended up with a big file, with a couple of years worth of random data from the GPS as well as the route I wanted. Bikely wants to use the GPX interchange format, but it didn’t want all my data – when I uploaded the whole lot it gave up in disgust. Eventually I figured out that I could load the data into Google Earth edit it down to just the route in question, then export it and upload to Bikely. That worked.

How do you publish a map like this?

So having made the map at the Bikely site you can capture it:

• Click on Share.

• Choose Display this map on your blog or website.

The result is a fragment of HTML you can paste into the source of a web page.

Only trouble is that if you are working in a word processor, and you paste in code like this all you get is ugly code in your document. Besides, the code by itself doesn’t give me the static version of the map I want for the print / PDF version of the document.

So, Ron Ward has extended the existing ‘embed’ feature in ICE, for embedding stuff like audio and video. To use it I have to do three things:

1. Put the Bikely on-my-site code into a file somewhere that it can be accessed over the web. In the future we’re going to make this really easy, so you don’t have to think about it, but for now I had to create the file and copy it to my website. (You also need to change the empty iframe tag to contain a blank comment because of a Firefox rendering bug).

2. Paste in a screenshot of the image, like this (on the Mac I hit Cmd Ctrl Shift 4, then selected the area I wanted to capture, and pasted into this document):

1. Link the image to the file. In my case the link looks like this:

2. http://ptsefton.com/map3.html?embed

3. View the document in ICE, and the HTML version is magically turned into a live map, while the PDF version, which you can’t see here, has a static image like the one immediately above.

Now, I know all that is complicated, but eventually we will automate the process so making a map is as easy as uploading the data from the GPS (which is hard enough on its own). ICE will automagically turn a tracklog into a map, and generate the printable version of the image for you. But this will not be a one-off development. There will be a plugin system so programmers can add formatters for lots of different kinds of data.

Issues

While this process works, it raises some interesting questions, for which I don’t have any answers:

• In a scholarly context, would it be OK for me to edit my GPS log, as I have admitted to doing here? Depending on what the map is for that might be alright, but there are some kinds of data that really should not be edited.

• If I expose the raw GPS data how do you know whether I have edited it? Would you trust it if it was signed by the GPS device so you could tell if I had tampered with it? Do you trust me not to hack my GPS device? Would you prefer it if I took along a witness? What about if you sent your GPS device with me, with a tamper-proof seal? Do I trust you?

  (A quick search GPS units that can sign their data turned up a patent, but it’s designed for shopping, not science 1)

• If I do expose my data, and link off to the the Google map how do I preserve or future-proof my publication? Can I put the data file in my institutional repository? What if it’s really big data, like the stuff2 the ARROW team recently added to the Monash repository? What happens when Bikely disappears, or changes its terms of service? (Something like AONS3 should hep here).

• How can we make the whole process work seamlessly so that working researchers to use it?

These are the kinds of questions the research and higher-education community will be looking at in Australia as part of activities going on around the new Australian National Data Service4 (ANDS).

On December 22^nd 2007 I officially became the Manager, Software Research and Development, in the Learning Futures Innovation Institute at the University of Southern Queensland.

This means I will (at least) look after software development for the ICE project, and for the OpenCourseware Initiative at USQ, and will continue to guide the repository work we started with RUBRIC.

In the short term this means a swing back to looking at how ICE can be used for courseware, continuing work on integrating it with Moodle both for USQ and for our Open Courseware project as well as pushing into new areas of e-learning. During 2008 we will also try to get involved in work on the new Australian National Data Service1, developing software and processes to help e-researchers integrate data with publications throughout the research lifecycle.

I’m honoured to be reporting to Prof Jim Taylor, who’s heading the institute, but I’ll also be sad that Prof Alan Smith is no longer my supervisor. I’ll still be looking to Alan as my guide through the jungle of university policy, politics and procedure.

I’m only back at work for 3 days this week, before heading off on a road trip to Kangaroo Valley so it will be mid January before we get the Software R & D team together as a group. Meanwhile there’s plenty to do getting ready for an important demonstration of USQ’s online infrastructure in February.