Tuesday, November 16, 2010

Visualization Links

Researching visualization software. What we will need for viewing object graphs and tree-structured data. Looks like there are some open source projects that ride on top of Java 2D, and provide visualization features such as layers, zooming, and panning. There are some open source apps built on top of the framework, so customizing may be straightforward.

This chemical model visualizer: http://sali.rguha.net/ depends on the object graph visualizer: http://www.graphviz.org/ which rides on this open source ZVTM project: http://zvtm.sourceforge.net/

The idea would be to have 2D, layered, multi-zoom interface into the trees of the tagonomy, so that tagonomies become visually recognizable and explorable spaces.

Thursday, November 11, 2010

Tagland on sourceforge.net

Please visit tagland.sourceforge.net to check out the source code for tagland. Under current development are some of the ideas sketched out in previous posts. Specifically, I've got a PropertyPage system working which allows you to view and edit Java Bean properties on any java.lang.Object. This will plug into the tagland command-line menu system to edit the Anarchia.us domain objects, similar to how NakedObjects does it, but much, much simpler.

These are baby-steps, but it demonstrates my bottom-up, Managing SimplicityTM, CDC approach. I'd love to hear your comments and contributions. Please feel free to join the sourceforge project here.

Friday, August 6, 2010

Nail 2.0

I live in Berkeley. Most of the houses here pre-date me. Almost all are built with wooden frames, which means almost all were built using nails. Imagine, if you haven't already, that the Naitional Nail Authority has issued Nail 2.0. Immediately, however, a bug in the Nail 2.0 implementation causes the underlying Nail 1.0 implementation to AB-END and fail to build. So immediately, of course, all Nail 1.0 and later spec nails cease to exist. So 95% of the houses in Berkely immediately fall over, and the town looks like the last little pig was smart after all, until owning a nice brick home in a town of wooden houses all broken up with gas lines open all over the place starts to seem like a really bad idea.

This kind of Scorched Earth vision was all the rage in the last part of 1999. I was working in Financial Services doing portfolio analytics server-side in Java with a legacy back end of millions of lines of FORTRAN code. So we didn't sweat the Y2K bug at all -- financial analytics really have to be able to go way into the future for Bonds and what-ifs, so the system had always been written to handle Y2K dates. But I remember thinking that if the EastBayMUD servers, which I had toured, and were highly COBOLized, were to crash, then for a few days we might not have water. Though more likely was just an EBMUD water bill for a century's worth of water.

So why don't houses fall down when they issue a new nail? Because objects in the real world are manufactured, and then installed and left there with no dependencies. Having a build system is a luxury deployed by just about every software shop, and so is having developers who use version control, test suites, etc. Yet, when we build software, we don't package everything up and ship appliances. We ship tiny bits of code that depend on lots of other code. And our bits of code were made by the interaction of thousands of other littler bits of code. And we send all our little pieces, with all their dependencies on pieces that we don't ship, out in their little boats. Sorry, lost the house metaphor for a boat reference. But I like the imagery of it.

In the dot-com binge, my brother and I started ToasterWorks.com and designed what is now called Cloud Computing. We designed, coded, networked, prototyped, and documented most of, say, what you can get from Amazon EC2. My brother worked on selling it to Dell, Compaq, and some large ISPs. We got really close, but we got beaten to market by several competitors.

I knew going into it that the ideas I had were the result of being in a computer geek zietgiest that probably included about 2 million really sharp minds. I knew there would be lots of others with identical startups, because appliances had already busted out, and linx had GUI terminal servers, so the concept of provisioning a blade at boot time with a networked version-controlled repository of boot images and large data mounts seemed like the way to go since it was clear that big iron would make some kind of comeback. I was also working then on designing web services so that all our virtualized boxes could have very simple duties and just network their messages back and forth. But like I said, that's no boast--lots of people were all scratching their heads and thinking about the future and how it might be manageable with all these dang Linux boxes stacked in the server room and all these dang software pieces.

So now we have Cloud Computing and we have Web Services. But we are still missing some of the things on my wish list.

Real packaging. You get the immutable bits that will work. All of them.
You can still have a build system for assembling and testing these beasts. But don't expect users to have a build system that can handle your kruft.

Simple communication.
All text communication.
Bi-directional communication supported, and one-way messaging is preferred.
To implement this today, I'm thinking: XML messaging. XML file formats. Use schemae and attach them to the document. These all seem viable and established now.

Real interfaces. Typed, inheriting, globally known, versioned, assembled interfaces. (Possible with things like XML messaging and JSON. But also perhaps with sending objects around.)

Come up with some kind of object that can be sent around the network and stored and versioned and transformed, round-trip, to text and back. Java does this, and it plays well with real object databases. I suspect Javascript could do this well, too, as my brilliant friend Brian hipped me to, but he said he detests blog technology, so I guess I shouldn't link to him. Anyway, he says it's totally doable, and folks are doing it.

But I haven't had too much time to see what is up in the javascript-as-object-nirvana world, since I am happy as a clam since I got the schema for the anarchia.us objects coded in Java and stored in db40 and accessed with the rudiments of Retro (my command-line framework) working on my laptop. Java objects. Simple. Not "Value Objects," but POJOs. Properties, collections, relationships--all store nicely in db40. And I'm not doing any kruft. No annotations, no JAXB, no mapping file, no injection, no nothing.

I'll put up the source very soon, and design docs. For now, please read the other posts in this blog to see where I'm hoping to go.

Wednesday, June 16, 2010

A day in the life in diorama

Let me tell you how I work.

I keep all my documents in a hierarchy of folders.
The folders are all named tersely, but human-readable and obvious.
I put files together that are in related projects.

I maintain this hierarchy of folders in CVS, which allows me to archive it, along with dated versions of everything I add or change in the hierarchy.
I buy time on a cloud server and run several processes, one of which stores my hierarchy, in sync, archived.

My data are protected by industry standard security which would be good enough protection against any except the highest level hackers with access to large compute clouds (e.g. the government).

OK. Everything above is factual, extant, and uses existing open source technology. Everything that follows in this post is a dream, combining how I work, and new, open source software to model and automate what I now do manually, and what I cannot currently do, but wish I could do.

In addition to this hierarchical, archiving, versioning, distributed file system, I maintain my metadata in a system called Diorama. Diorama stores all the information about my files, and maintains this as a context that can be searched, executed in, added to. I can choose to add new items to my own Diorama-run file system, or I can add new items that are stored out in the cloud, in which case Diorama simply stores the URL and any metadata that get associated with that document. I use the Diorama-run file system for text and small images, and remote storage for larger, binary files. You could use any file system you want, and just have Diorama maintain metadata and indices to documents only.

In addition to tags and categories, where you were in the context at the time of document status change, can be saved in the metadata as well (if you allow your context history to be saved in metadata or metadata links to your context history database). Diorama uses hierarchies of tags as categories, with paths, in "tagonomies", or, tag-based taxonomies. The taxonomies are only suggested, and there are so many aliases, mispellings, case-changes, that they cannot be said to be purely concrete taxonomies, since they don't root together in a particularly organized or distinct way, and in many ways are really graphs. But Diorama soldiers on through the aliasing process, until small set of commands and small set of paths are what I work with most of the time, and those paths and commands become familiar to me and to Diorama, so they become quicker to execute in the command shell.

Since I work mostly through Diorama, everything I do is stored in my personal command history database. It is indexed by time. Documents and remote processes can link to the exact time I executed something, and for API calls inbound, any Diorama context I was logged into can always return its local timestamp, so that the history data are kept local, but can be indexed by request in the future.

Any time I am using Diorama, I get to keep history. So this works regardless of where I log in from. The history data are kept in my Diorama distributed database.

The Diorama command line is simply a command line environment that has a display area that can show menus of commands, pictures, web pages, etc. Diorama has a context area that shows information about the context you are in. Menus, commands, resources, shortcuts, notes, etc. You execute commands to open documents, take actions like send mail, create content, publish, and subscribe.

The context area shows you the tagonomy of where you are, and also the tagonomies of any document you have open, etc. So for this page I might see
"docs/dioramaBlog" or "projects/diorama/blogs/InThroughTheDioram". You can tag anything with these path tags (the character I use is a / but you could use anything, such as "projects :: blogs :: In through the diorama"). The system treats things in the path as parent contexts, and learns about how you categorize things this way. It starts suggesting context paths, for example. The system also does quite a bit of aliasing, phonic checking, and case-insensitivity.

Diorama also has a context area that shows you where you are in the web of contexts. Most of the contexts you will spend time in are yours, like your home directory, or directories of projects you are working on. But you stay in these contexts in multiple windows, and can switch contexts with the keyboard, while still in command mode on the command line. It's very much like being in emacs, sending mail. But it is very menu driven and simple, obvious, and fast. But you will be able to visit contexts of others. You will be invited to join these contexts, because a person on the other end knows you somehow. The system is based, just like human interaction in the offline world, on trust. Most people don't trust pimps, hookers, or car salesmen, but in fact these people are at least honest about their game. Still, when you go to hang out in the park with your best girl, you don't want pimps there, and when you go have a drink with your friends, you don't want a used car salesman standing over your table like a drunken mariachi. So Diorama has no ads. And everyone you run into, or whom you allow access to, is known to you.

Sunday, June 13, 2010

Tagonomy Display; features; attributes of tagonomies

See also: Why diorama?

Supposing that we have a definition of a tagonomy (elements, paths, relationships, aliases, semantic meaning versus object storage), then we can display them.

I have two user-interface rules that apply here: 1) information for navigating the system should be accessible and distinguished; 2) the signal-to-noise ratio should be very high.

For visualization, imagine you are wandering around in a world of your favorite zero-information background, and in front of you are high-information tokens, and around you are related token which could be useful if you wish to move towards them. I could see this as a virtual reality walk through a park with some avatars, etc. But for the current proposal, let me have you visualize something a bit more IBM-Gothic.

For Retro, you have a lovely black background, and words in pleasing fonts and readable, gay greens and yellows and whites (or whatever color stylesheet turns you on) get closer and farther away as you get close to them semantically, so that you see the word "eMail" in bright green letters, and it is significant. Or, if you are in the middle of navigation or choosing items from a menu, these are the only objects/words/pictures that are really highlighted and green.

In this space, we are navigating the metadata, the context. We are not storing the objects, we are providing all the information about the objects, the relationships.

- by year
- big font for years
- project inception dates, meeting person dates, most highlighted/visited/priority get biggest font
- by category
- by priority
- by tagonomy and by date as layers

Above a certain size, every object in the space, with its own representation gets its representative node to bump its display size. So each folder in the space has a template or stylesheet that exports size displays.

Displays for an object in a tagonomy:
  • At very small, it is just a word.
  • Next, it is a series of words, and maybe an icon, if icon display layer is on.
  • Next, it becomes a description paragraph, extract, abstract, with picture, like an article or summary.

See also:
Why diorama? Why now? And what is diorama anyway?

Why diorama? Why now? And what is diorama anyway?

VoodooPad was *almost* the right application.

Lotus 1-2-3 was almost the right interface for a very right application.

WordPerfect 5.1 was the right interface, but it was only a word processor.

The right interface is a command-line, with a menu, and a display.
It's not that we evolved past the command line; it is that we failed to evolve the command-line.

The modern interface is retro.

Retro :: a command line interface ...

  • that connects you to all personal data, projects, thoughts
    and remembers where you put things

  • that connects you to real people you know
    and no one else gets to look at that information
    and you don't look at any ads.

  • that lets you categorize, file, and find things naturally
    in a system that matches how we think and ask questions
    and not at what is most efficient for the computer or designers
    of input forms or advertisers.

The new interface is mostly non-information. The information sticks out because it is information.

There are no ads to masquerade as information. The menus can be navigated by picture, text or both.

Each menu level is a space, it remembers when you were in the space last, your preferences, etc.

If you are invited into the space, or it's your space, then you can run certain menu items in that space, such as checking eMail, messages, comments, etc. Other spaces might be discussion forums just for people who are invited into that space. You also have public spaces. These get mapped to HTML for delivery on the WWW.

You can set up scripts and commands at each menu, in each space.

You can link to data storage for any document that has a valid URL.

You can write notes anywhere in the system and tag them with any tag, alias, or path.

There are some global tags (always on your shortlist) that get you to buckets, or inboxes.

These are tags like "soon" and "todo" and "high priority" or "ideas". These are simply your most used tags, and also tags that map to some future search or workflow.

Ultimately, this system should be used to run tagonomy and anarchia. Anarchia is a social network built around interactions with people as artists. Each person connects to another's art/expression by linking themselves, or to a specific piece of art.

We are about breaking down the definition of artist. Everyone is an artist. Art is the expression of ourselves and our experience through a medium to another person. It is successful art if it conveys expression better than the workaday ways we have to communicate and be understood.

Some names for this stuff:
  • Retro
  • TagLand
  • diorama
  • Tagonomy
See also: Tagonomy Display