FTP ASCII unmangler

FTP text mode is evil.

I made the mistake of transferring several important binary files from OS/X to windows last night, using FTP. Actually, a few mistakes were made along the way. 1) I didn't check that I was in BIN mode first, 2) I didn't verify the integrity of the files after the transfer, and 3) I deleted the sources.

Luckily, it was Unix-to-Windows, which means all #10 octets were replaced with #13#10. First I tried dos2unix with no luck. Then I wrote a program to replace all #13#10 sequences with #10 and crossed my fingers.

It worked. Here it is in all it's inefficient glory. Maybe this will help someone else someday. No guarantees, but it's worth a shot if you're desperate.

import java.io.*;

public class Unmangle
{
    public static void main(String[] args) throws Exception
    {
        InputStream in = new FileInputStream(args[0]);
        OutputStream out = new FileOutputStream(args[1]);
        int prev = 0;
        int b = in.read();
        while (b != -1)
        {
            if (prev == 13 && b != 10)
                out.write(13);
            if (b != 13)
                out.write(b);
            prev = b;
            b = in.read();
        }
        if (prev == 13)
            out.write(13);
        out.close();
        in.close();
    }
}

Social Bookmarking == Free Metadata

Metadata is expensive. Librarians aren't the only ones privy to this fact.

I remember the pain we went through in bringing HP's FTP site to the web. The first step was converting the old README files to HTML. Perl made this a snap, but somehow it didn't address the now-more-apparent quality problem. So we slurped it all into a Paradox database had a big metadata entry party.

Ok, the word "party" might be a stretch. There was technically pizza involved, but it was more of a bribe. It lasted days, and nobody really celebrated 'till it was over.

I distinctly remember the phrase "metadata monkey" entering my vernacular at that point.

I don't have anything against monkeys. Monkeys are cute. Monkeys at keyboards are even cuter. But metadata entry has long been viewed as a thankless job.

Now, sites like del.icio.us have figured out a way to get metadata monkeys to work for free. The incentive? Not pizza, not even bananas: just the ability to store our own descriptions, share those descriptions with others, and access it all from anywhere.

It makes me wonder about the role of the library in creating authoritative, versus curating social, metadata.

Resources, Representations, Repositories, and RDF

Last week, Carl Lagoze gave an update on the OAI-ORE work at Open Repositories '07. ORE is a new project that intends to specify how heterogeneous repositories can exchange information about the digital objects they hold. Although they're not necessarily going after a new protocol, I still think of it as taking OAI-PMH to the next level. It's not just about metadata anymore.

For me, the most interesting parts of the talk were webarch-related. It all started with the statement (to paraphrase) "we must build on the web architecture". Carl then pointed out how representations are essentially second-class citizens on the web.

That got me thinking. At the most basic level, repositories are all about managing bitstreams (whether they're considered data or metadata). In webarch, bitstreams seem to equate to what they call "representation data". And a representation is defined by how it relates to a resource:

"A representation is data that encodes information about resource state."

So, in w3c-speak, a repository manages representation data. Okay, that's just a terminology change. But what about this statement:

"For robustness, Web architecture promotes independence between an identifier and the state of the identified resource."

That makes a whole lot of sense for the web when you consider how often web pages change. But what does it mean for repositories? How do we manage bitstreams if we can't identify them? The answer must be one of the following:

• Indirect identification. Identify the associated "resource" in order the work with the bitstream(s).
  
• Reification. Elevate the bitstream to a "resource" so we can talk about it.
  

How about if we want to model the repository as an RDF graph? Well, we know that representations can have metadata in addition to the payload. So in order to do this modeling, we need to reify. Internal to a repository, representation triples might look something like:

representationA represents urn:example:someTextFile
representationA contentType "text/plain"
representationA payloadLocation "/path/to/someTextFile.txt"

I think the OAI-ORE work is going to attempt something like the above: a model (and maybe a format?) for expressing resource-representation information in a repository-neutral way. It will be interesting to see what pops out.

A good source tree

A good source tree:

• Has a root-level README file.
  
• Contains all compile-time dependencies.
• With versions noted.
• With originations and licenses noted.
• With transitive dependencies noted.
  
• Can be compiled and linked easily.
• With one command.
  
• Without a network connection.
• Can be unit + integration tested easily.
• With one command.
• Without a network connection.

Hardly coding

URGENT ASSISTANCE!!!

HELP I CAN"T COMPILE>>> ERROR!!

[checkstyle] C:\work\mptstore\trunk\src\java\org\nsdl\mptstore\query\
provider\TriplePatternSQLProvider.java:35:45: '3' is a magic number.

I EVEN TRIED INSTRUCTIONS ON HOW TO PROGRAMMER??

http://secretgeek.net/howtobeaprogrammer.asp

SO I SERCH GOOGLE LIKE A GOOD PROGRAMMER AND FOUND THE ANSER:

http://www.youtube.com/watch?v=yPzAjiLr5Zw

BUT HOW TO I PASTE VIDEO I"M USING ECLIPSE??

THANK YOU!!!

Reducing Firefox Memory Consumption

Firefox uses incredible amounts of memory. Some of this is caused by various undiagnosed memory leaks. Looking around on the net, I found a few things that can help, short of restarting all the time:

1. Make sure you have the latest version (currently 1.5.0.5)


2. Enter about:config in the address bar.


3. Right click, add Boolean: config.trim_on_minimize (true)


4. Right click, add Integer: browser.cache.memory.capacity (32768)


5. Restart Firefox.


6. If things get sluggish, minimize the window and restore it.

Using a tool like CachemanXP can quickly show how well the "trim_on_minimize" setting is working.

DTP Model

The X/Open DTP Model is a widely-supported standard that defines components and interfaces useful for dealing with distributed transactions.

In this model, the application uses a transaction manager to help carry out a global transaction. The app communicates with the transaction manager via the TX interface, which provides the transaction semantics (begin/commit/rollback/etc).

Each data store involved in a global transaction is exposed to the transaction manager as a resource manager. The communication takes place over the XA interface, which defines functions for lower-level transaction semantics. The transaction manager uses the XA interface to carry out two-phase commits, and the resource manager uses it to dynamically enlist specific resources in the transaction.

References:
Nuts and Bolts of Transaction Processing, by Subbu Allamaraju, subbu.org
Global Transactions - X/Open XA - Resource Managers, by Donald A. Marsh, Jr., Aurora Information Systems, Inc.