Tuesday, November 30, 2004

Campaign to Add foaf:mbox_sha1sum to Flickr APIs

One very successful way to smush [1] FOAF is to match on the foaf:mbox_sha1sum. Many FOAF producers, such as LiveJournal put foaf:mbox_sha1sum in their FOAF. Nice!

Unfortunately, Flickr does not support FOAF. OK, no big deal. I made FOAF wrappers for Flickr people profiles. However, if I am not a contact of that user (which for 99.99% of Flickr users, I am not) I can't see their email addresses. This is a great idea, I'm glad they don't publish it to everyone. Except even in the Flickr APIs there's no way to get the email as a sha1 sum.

I sent the Flickr API mailing list a note asking them to add this. How can I make a stronger case? They don't even need to adopt RDF or FOAF here. Just put in a sha1 hash of the user's email address in the response to the call flickr.people.getInfo.

That would be amazing if they did, and smushing all these FOAF profiles would become possible.

Advice on how to make the case?

[1] To combine RDF triples so that duplicate nodes are removed and similar objects are consolidated.

FOAFverse Coming Along

The latest on the FOAFverse:

I've refactored the downloading of the FOAF file to use HttpClient from Jakarta Commons. This gives me much more exact control over the HTTP connections (setting Accept header correctly, setting User-Agent for LiveJournal doesn't get pissed) and I can easily add caching (thanks EHCache).

I've created a HttpClientTemplate to get all the boiler plate code out, like handling exceptions and setting common headers.

My queue is in place to fetch new FOAF files as I discover them. Which leads me to my next favorite refactoring: abstracting out the Strategies for finding the Friends of a Person. There are many ways to link to the FOAF file of a Friend. A common way is rdfs:seeAlso. Others might use the foaf:PersonalProfileDocument or its inverse foaf:hasPersonalProfileDocument. Each method of finding the FOAF file of a friend is a Strategy. As we run each one against a FOAF file, we find Friends' FOAF files, and put them in the queue to be downloaded and searched.

We're getting there. Luckily this is all code that is separate from the webapp I created the other day. It just displays the FOAF in human readable ways.

#rdfig is now #swig

Semantic Web Interest Group IRC Scratchpad


eg. I've just (after logging in with "/msg nickserv identify ...") done "/msg nickserv set url http://danbri.org/". It's easy. And fun. And educational. Msg nickserv today!


I've just registered my nick "sethl" to the url http://www.picklematrix.net/.

So go on with your bad self, DanBri, start scuttering!

Also, might as well create HTML links from nicks in the Semantic Web Interest Group Scratchpad logs using this new URL info. Or, first get my FOAF, and use that link.

Sunday, November 28, 2004

Began Foafverse

Tonight I began work on Foafverse. Foafverse is my answer to FOAF Explorer. FOAF Explorer is nice, but works via XSLT. RDF's serialization into XML is messy at best, and never consistent. The XSLT sheets from Explorer seem to work OK if all FOAF is formatted the same in XML. RDF in XML, as we all know, can't be predicted. This serialization shortcoming is a minus (minus in that RDF as XML can't play well the the zillions of XML tools and pipelines out there).

Anyhoot, to properly do a RDF powered site, we have to work at the Graph level. So Foafverse uses Jena so that all work is done with graphs and triples. As long as the FOAF can be parsed as RDF/XML, then we're cookin' with statements.

For extra buzzworthy-ness, Foafverse uses Spring Framework to wire everything together. It also is helping as I create nice Bean utility classes for working with Jena.

What will Foafverse do? Merely load in FOAF, save it, periodically refetch it, and query it. But working at the graph level is very liberating.

Done: Loading of FOAF into Jena and its persistent (into MySQL) Models. Querying the FOAF for persons. Webapp framework built, ant scripts, Spring application context, deployments, etc. JSPs, controllers, facade layers, etc all exist now.

TODO: shucks, tons. First up: smushing based on RDFS for FOAF plus extra OWL goodness. We have all this FOAF now, let's place on RDFS and OWL to connect the dots. Stress test Jena's support for inferencing.

Friday, November 26, 2004

My First MovableType Plugin

I wrote my first MovableType plugin tonight. It's the MTSha1 Plugin. It allows you to generate a SHA1 sum of the body of the <MTSha1></MTSha1> tags. I created this to generate foaf:mbox_sha1sum properties for comment authors. And the best part is, I'm pretty sure it's working correctly. :)

Better Comments RDF for MT Blogs

In the never-ending effort to get more triples into the world, I've cleaned up the comments RDF that can be found in my RSS 1.0 Feed.

Now, the authors of the blog comments are foaf:Person. For extra smushing pleasure, we also have a foaf:mbox_sha1sum of the author. So we should now be able to tell, through an InverseFunctionalProperty, who exactly wrote the comment.

I had to add a new property, blogcomments:hasComment, to link an entry to a comment.

Comments? Thoughts?

Use Case: "Allow people to comment on my blog who have successfully commented on blogs found in PlanetRDF.org"

Upgraded to MovableType 3.121 and TypeKey

Spent tonight upgrading to MovableType and TypeKey.

I finally got fed up with deleting spam comments, even with MT-Blacklist on. In one day I had over 100 comment spams.

So, with TypeKey, that should all stop. I feel bad for those that want to leave comments, but don't have a TypeKey account, though. Comments are what make blogs interesting (otherwise it's just Soapboxlog). So hopefully TypeKey has critical mass to use.

I've thought about How FOAF and SemWeb can help the blog comment problem before. Not sure if that's relevant anymore. I would hope that all this FOAF and semweb technologies could help to solve this problem.

Anyhoot, looks like we're back in business. I still have FOAF linked to every page. And my RSS 1.0 Feed still has RDF triples describing the comments. I think I need to update it now to include FOAF.

If other blogs included RDF about comments, and their authors, we could get a real trust network going with a context of "allowed to post in blogs".

Wednesday, November 24, 2004

New FOAF Property : foaf:hasPersonalProfileDocument

While there exists a way, in FOAF, to say "There is a Personal Profile Document that has a primary topic of XXXX Person", there doesn't seem to be a way to say an inverse.

I wanted to say "Person XXXX has a Personal Profile Document at http://www.example.org/foaf/392j92j3".

There is a foaf:primaryTopic property, but there isn't an inverse. Also, it's domain is foaf:Document, which is too broad. I really want to link a foaf:Person to a foaf:PersonalProfileDocument.

So I've embedded the following RDF into all the Flickr FOAF:


<rdf:Property rdf:about="http://xmlns.com/foaf/0.1/hasPersonalProfileDocument">
<rdfs:domain rdf:resource="http://xmlns.com/foaf/0.1/Person" />
<rdfs:range rdf:resource="http://xmlns.com/foaf/0.1/PersonalProfileDocument" />
</rdf:Property>


Thoughts?

Better FOAF For Flickr

After some advise from Danny and Phil, I've reworked the Flickr FOAF a bit. I've had to pull info from three different URIs (including their Flickr Services) to assemble this FOAF.

FOAF for Flickr User Seth Ladd

The major new differences:


  • nick

  • name

  • complicated, but accurate, link to user's Flickr homepage. This is now a IFP

  • more links to Flickr and its homepages

  • no rdf:about for the actual foaf:Person Resource. Is this correct?

  • foaf:PersonalProfileDocument elements to link to other FOAF



Wish I had access to the emails, or at least the mbox_sha1sums, to add yet another IFP. So far we have just Flickr's Profile page, but foaf:nick isn't too bad.

Adobe XMP Specification

Just a note for me to remember, but whatever RDF/XML Flickr might choose to export about its photos, they should use the Adobe XMP Specification.

Looking for a New Blog System

I received word that my current blog setup can't take comments (of any type) or apparently TrackBacks. sigh. In my quest to rid spam in my blog comments, I managed to kill discourse. Hope that's not an omen!

I think it's time to upgrade this blog system. Any suggestions? Not that my comments system will allow feedback. :(

My requirements:


  • A wiki-esque input system, to rid myself of typing tags.

  • Good SemWeb Support: FOAF and RDF of everything.
  • Good Spam Filtering

  • The usual: comments, trackback, categories, etc

  • Simple Setup

  • Import of all my current MovableType posts



In the meantime, off to fix comments for the non-spammy.

UPDATE: I removed the uber-greedy spam filter regex that was causing the problems. Because emails are required, they were setting off the alaram. Good thing I'm in Hawaii, where we love Spam.

Danny Read My Mind

Danny Responds to Flickr FOAF

Danny read my mind. I was wondering how best to say, in FOAF, "This is a person, who has a homepage at Flickr". I didn't want the object of the <foaf:Person> statement to be the URI to the Flickr User's homepage. Then it makes it sound like the homepage is a Person.

Danny thought outloud:


Person serviceID _:x
_: service http://flickr.org
_: serviceName "flickr"
_: identifier uri


So, the question is... what is the object in the statement for <foaf:Person>? A URI to a person? Do people have URIs? Or just to a resource that represents the person? In which case, the Flickr profile page would be good enough. And, luckily, through some OWL and mbox_sha1sum, we can uniquely identify the person regardless of their URIs.

Tuesday, November 23, 2004

Generating FOAF From Flickr

Thanks to my horrible PHP skills, I can now generate some FOAF from Flickr profiles.

For example, here is some (very basic) FOAF for a Flickr Member.

Ironically, Flickr's Service API can retrieve a user's information, but almost nothing is in there. Plus, you can't tell what groups a user is a member of, or what contacts a user has.

My end goal for this is to generate FOAF with name, nick, gender, location, groups, contacts, and photos. Then we'll start on RDF for the photos themselves, including the tags a photo has.

Then we'll scutter it all, and slap some ontologies down. Should be interesting!

Question is, how good are my screen scraping regex skills?

Monday, November 22, 2004

FOAF, Please

Flickr: Flickr Forums : FOAF, Please

Stewart, from Flickr, chimed in to the discussion on adding FOAF to Flickr's pages. My response:


As for FOAF, I really want to explicitly set my FOAF file in my profile. I don't want to add it into my description, because then finding it is more difficult. So, as you say, a text field where I can point to my FOAF would be perfect. Of course, if I don't specify my FOAF, then Flickr would create a simple FOAF for me (just w/ name, nick, pointer to Flickr homepage, and sha1 of email addr).


Adding FOAF output of a Flickr user's profile seems like a really easy thing to do. Then, we begin to use that to help with answering the "Who" part of queries.

Thanks Stewart!
Seth

Get Your FOAF Noticed

crschmidt - Recent Entries

Chris Schmidt writes:


Send it in email - If you have a client which allows you to edit the email headers, you can include an email header X-FOAF: http://example.org/foaf.rdf that indicates the location of your FOAF file. Tools which understand incoming mail may be able to use this header to find information out about you, helpful for whitelisting and other purposes.


He lists other ways to get your FOAF out. Sounds like a good idea, especially for mailing lists that archive the emails.

Thursday, November 18, 2004

Tagging Free For All

What if anyone could assign tags to photos in Flickr? Also, what if they can already and I just can't find that feature? :)

How much of the metadata free-for-all would be "good" metadata? Where good is defined as useful. Is there such a thing as useless metadata? I argue there isn't, as long as you know the context. For instance, if you know who (as in, FOAF) assigned the metadata (using the word tag from now on) then it doesn't matter what the tag is. The whole point of the semantic web is to allow anyone to say anything about anything. Trust networks emerge, either implicit through certain algorithms, or explicit through group memberships. These trust networks allow weights to be associated to tags. Therefore, the same tag can be both useful and useless to different people.

Due to the use of URIs, external systems can be built to created these Tag free-for-alls. Flickr is very close to allowing URIs and RDF graphs between the page about the photo and the photo itself. Once that happens, and they start exposing the tags in the RDF about the photo, we are in a position to build the tag free-for-all.

The cost of generating metadata is very high. By distributing the workload across everyone who views the image, much much more metadata is generated.

I think the minimum step would be for Flickr to publish some RDF for the photo image itself, from the photo's homepage. Once that is done, these external systems can be built.

Tuesday, November 16, 2004

Flickr Forum Posts Roundup

I've been posting my ideas for more and better metadata over in FlickrIdeas Forum. Flickr is so ripe with metadata, I'm getting really excited about the possibilities we could do with it. I'm sure that by exposing the metadata as RDF (and formatted such that smushing is straightforward), the power of Flickr increases tremendously. At the very least, as new applications are built to take advantage of the new metadata (Show me all blog comments made in the blogosphere by authors who shoot on Nikon D70 and that have images on Flickr rated greater than a 3 out of 5 and tagged as 'dog'.) more people will see the power of metadata and tags, and add more and more tags.

Anyway, stepping down from that soapbox, here's the current roundup. Please post your ideas and followups!


  • Explicit Link to Photo in Feed Entry - Provide an explicit link to the image itself, not the page about the image.

  • FOAF, Please - Use FOAF, among other things, as the metadata for the author of forum posts, images, and image comments. Use foaf:mbox_sha1sum for easy identification.

  • Nested Albums - I propose adding more meaning to tags, such as subtypes.

  • Metadata for the Image Itself - Adding a <link> pointing towards RDF describing the image itself, especially its tags.

Monday, November 15, 2004

Adding Metadata to Flickr

I've looked into Flickr, and it looks like a promising source for metadata.

It turns out they do a pretty good job to start. They expose feeds for photos tagged for "animal". They also expose a feed for each user.

Here's where Flick need to go next:


  • Add <link> to feed for tagged photos in the HTML. For instance, in http://www.flickr.com/photos/tags/animal/.

  • Add FOAF output when viewing a user's page as a <link> in the HTML

  • Add RDF <link> for each picture. Expose EXIF and tags for each picture. Add metadata for author, and even who has marked it as a Favorite.

  • Stop using that awful Creative Commons RDF inside comments inside HTML. We've grown up, it's time to use <link> to bind CC licenses to pictures. (Kudos for using CC in the first place!)

  • Expose the CC license used in the picture RDF.



Flickr has a great opportunity for publishing a tremendous amount of metadata, generated quite easily. I've posted to their FlickrIdeas forum about Adding FOAF Export and Explicit URI to Image in Feed Entry. Post some comments there and let them know we want more metadata!

I mostly want to be able to browse to a picture, say, http://www.flickr.com/photos/gauviroo/1501146/, and be able to <link> to the RDF for that picture, including its tags, exif, and link to author's FOAF.

Disclaimer

I'm probably required to say that the views expressed in this blog are my own, and do not necessarily reflect those of my employer. Also, except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 3.0 License, and code samples are licensed under the BSD License.