Category: oldblog

What’s it all about, Alfie

I’ve just launched a new tool at Hatmandu.net, a text content and keyword analyser – in theory useful for search engine optimisation, but also to get the general gist of a text.From the notes:

This text content and keyword analyser is intended to give a more precise indication of a text’s most important words than other tools available. Most keyword analysers use simple word frequency (which is also shown here anyway), but that doesn’t relate the specific text to the language in general – common terms such as ‘people’ and ‘time’, for example, appear in many documents, but do not necessarily indicate the essence of the particular text being analysed. This analyser uses the TF-IDF statistical method to relate the frequencies of words in the specific text to their general frequencies in the British National Corpus. I am indebted to Adam Kilgarriff‘s version of the BNC, which I have adapted considerably for this tool. This analyser mainly uses the nouns in the BNC, on the basis that these are the parts of speech that best indicate the subject matter of a text. (At some point I hope to produce a version using an American English corpus, though I’d be surprised if the results were very different.)

It works with Twitter accounts (though it only reads the last 200 tweets, which may not form a usefully large body of text), and URLs where my humble scraping tool is able to extract the text successfully – most useful is the ‘paste text’ field, which will accept up to 1Mb of text (about 200,000 words) – so will analyse entire books if desired. Livejournal users can enter their URL (http://username.livejournal.com) assuming their account is public.

It’s a bit experimental at the moment, but hopefully might migrate from ‘possibly fun’ to ‘possibly useful’ in due course!

Posted on by Andrew Chapman 2

The narrative of illness

So, yesterday I was felled by illness. The night before, I lay wake hour after hour, aching and uncomfortable with stomach pangs. As the day went on, I felt worse, with hot and cold flushes, more pangs, total exhaustion, and I crept back into my bed for much of the day for further fretful sleeplessness. Even one of usual salves – watching one of the Peter Sellers Pink Panther movies – failed, as I just couldn’t concentrate. Inevitably, feverish thoughts roved to whether I had the dreaded swine flu.

Today, the day began with some queasiness, but as time has gone on I feel immeasurably better – I’m chipper, punning and have a renewed bounce in my step. Whatever battle my body was fighting, it reached some low points but it eventually won.

Which is what made me think of the parallel with narrative. Kurt Vonnegut said all stories boil down to ‘Man in a hole’: “Somebody gets into trouble and gets out of it. People never get tired of this.” Legions of Hollywood screenwriters (eg Blake Snyder, whose Save the Cat! book is quite interesting – and I’ve only just discovered he died a few weeks ago; or Christopher Vogler, who applies Joseph Campbell’s ‘hero’s journey’ analysis of myth to blockbuster movies) have made a career out of amplifying Vonnegut’s summary into detailed scene plans for film scripts. Everyone knows there are only three, seven, 20 or 36 plots (or eight, nine, 37, 69…) – or just one, really.

All of life is full of these little mini-dramas, overcoming challenges, confronting enemies, battling illness. It’s no bloody wonder we like stories so much – especially the ones where we win.

Posted on by Andrew Chapman 4

A new look at the publisher’s lunch

As usual, everyone’s talking about how publishing can survive, and how to make money on the internet. Paul Graham has written an excellent essay, Post-Medium Publishing, where he observes that it is wrong to think publishers sell ‘content’ – rather, they sell a means of distribution, and prices are dictated by that (ie, historically, the price of paper and printing) – if t’were otherwise, we’d all pay vastly different sums depending on the quality of the content. And we don’t. Bottom line: “Whoever controls the device sets the terms.” Prospect Magazine, commenting on Graham, also reminds us that we’ve seen all this before, back in Shakespeare’s time.

Meanwhile, Steve Outing warns that ‘Your news content is worth zero to digital consumers’, and that money is again in delivery systems such as neato iPhone apps. (He quaintly goes on to suggest micro-rewards – tip jars 2.0, I guess.) Jeff Reifman has weighed in against Outing saying ‘Micropayments could save journalism’. It’s hard to see how: if the headline writers are any good, the headline is where the news is – the rest is elaboration. I get my news from a few simple sources, all of them essentially ‘headlines’:

  • A few snatched moment’s of Radio 4’s Today programme between bouts of baby care – I really just get the 7am headlines
  • RSS feeds from the BBC and the Guardian on my iGoogle page – I’ll occasionally click through if I want the detail or I’m piqued by something
  • Twitter feeds

I buy one newspaper a week: the Saturday Guardian. I do read the news in it – but almost invariably I’ve seen it the day before on the web. I like it for the columnists, the features, the magazine, basically as a ritual entertainment to accompany a cup of tea. My wife just does the crossword. The physical newspaper, in other words, has become an entertainment channel rather than a news one.

Micropayments? I can’t see myself paying for news stories. Features… maybe, if they’re really going to interest me. Academic papers: possibly, if I’m researching something. That said, I did make one micropayment this week: we were planning to buy a new car seat for the baby, and only one place, Which, has a decent, up-to-date review of best buys, focusing on safety (ie there’s an emotive imperative here – and the possibility of saving money, I guess). They charge £1 for a trial subscription – but then sting you with monthly payments several times that. You can cancel any time, so I will cancel straight away. It’s very annoying: I just want one article, which I probably would have paid £5 for, simply because it’s not possible to get this quality information elsewhere. I subscribed because I’m bloody minded enough to remember to unsubscribe – though of course their business model partly relies on people forgetting, or being sufficiently charmed by the dull magazine you get in the mail.

Paul Graham says that the only kind of information people will pay for is that “they think they can make money from” – I’d add that saving money (assuming more is saved than the information costs!) might be a motive, and niche issues such as the baby safety report I mentioned.

Graham reminds us, as people like Chris Anderson have done before, that something else people will pay for is live entertainment. I wonder if this connects to another constraint upon pricing for publishing models: it’s noticeable that novels, DVD rentals, cinema visits, CD albums, all generally fall within the £5 to £15 range: people will only pay so much for entertainment that they know can be reproduced. Live entertainment, such as a theatre show, opera, music gigs and a decent meal at a good restaurant, is more of a one-off experience, and commands more value. In his excellent book 59 Seconds, Richard Wiseman points to research showing that people’s happiness is improved significantly more by experiences than by products. There’s no such thing as retail therapy.

Again and again I come back, too, to the feeling that modern content producers – writers in particular – have unrealistic expectations of fame and fortune. Most people don’t want their content, and won’t pay much for it even if they do. As Prospect says, we’ve gone back to a pre-Romantic time (I’m thinking of poets and gentleman publishers such as John Murray here, which is where the modern author-publisher dream of the last 200 years began) where writers have to work hard, diversify, hawk their products themselves, and not just sit back and expect a publisher (whose grip of the medium is now somewhat buttery) to make them millions. The Dan Browns and J K Rowlings are the lucky exceptions.

I’m a writer myself, so it’s not like I don’t have an interest in these issues – but I just write to commission, content I know someone seems to want, rather than trying to sell my own ideas, as the latter is so much hard work (obviously I thank my stars for those commissions – and make most of my money by doing design work anyway – ie making vessels for others’ content). Whatever ideas I have (mostly daft, I admit) I give away for free, often at this website.

Perhaps the answer lies in Kevin Kelly’s 1000 True Fans argument: build a core, devoted audience – if your stuff is good enough (and has a bit of luck and a fair wind), there will be some people at least who will go to your every gig, buy every T-shirt, read every book. If you can’t find 1000 true fans… maybe it’s time to be honest and admit the world isn’t knocking at your door. Do something for free. See what happens. Oh, and go out for a nice meal: it will make you happy.

Edit: After a challenge on Twitter to crowdsource payment for an article, you can now pay micropayments to get me to write an article on ‘The Modern Ninja’! I can’t lose: if not enough money is raised, it proves content isn’t worth much to people (well, er, my content…); if it is, I get a paid commission! (Oh, and if less than $300 is raised, I’ll refund your money folks!)

Posted on by Andrew Chapman

The nonsense of an ending?

I’ve just finished watching the third season of Heroes. I enjoyed it, but various things about it – and about Lost (I’ve yet to see season five of that, though), and other contemporary TV shows, make me ponder about narrative theory. As one does.

One thing that’s really noticeable about these series is their reluctance to let characters die. In Heroes, the same core of characters continues from one series to the next, and various ingenious ways are thought up to aid this, to the extent that they can even reappear after death, whether as a figment of someone’s mind, or as a physical duplicate, or in someone else’s body, and so on (no names to avoid spoilers). The actors must have really good contracts drawn up… Yes, a few loveable characters have died, but they’re the exception.

A similar pattern persists in Lost, which seems to throw Occam’s razor ever further to the wind: it relentlessly multiplies entities beyond necessity, beyond the enjoyable teasing of the audience to the extent of suggesting the writers are rudderless. Season five, I’m told, may change this view – we’ll see.

Much is made of the ‘story arc’ these days – how TV shows have become more sophisticated, and demand a complex level of attention. Which is fair enough, and of course books have run over multiple volumes before – but I wonder if the arc is being stretched to breaking point, and sometimes misses a fundamental of narrative: the expectation of an ending.

Frank Kermode, in The Sense of an Ending, wrote that fictions (as with human lives) have an implied ending all along, which makes ” possible a satisfying consonance with the origins and with the middle”. Peter Brooks’ Reading for the Plot also studies how we “strive toward narrative ends” – he coined the phrase “the anticipation of retrospection” for that sense of how we imagine ourselves at the end, looking back on where we are now.

We are promised an ending for Lost in season six – but is there any way we can meaningfully look forward to it? What about Heroes: we’ve saved the cheerleader and saved the world a couple of times already – what’s left? It just doesn’t seem clear that there’s a narrative architecture any more. Maybe they’ll have to end, like Conan Doyle’s Sherlock Holmes stories (another character brought back from the dead to satisfy a hungry audience) with a whimper more than a bang.

Another TV series that comes to mind is Doctor Who – long ago this came up with a clever notion for letting the character die, but the series live on: regeneration. We want the Doctor to keep having adventures – but even he is mortal, and the 12-regeneration limit gives a whiff of the grave that helps keep his adventures alive, I think. But I bet if the series is still running, the BBC will give in to the temptation to renew his regenerative lease when they run out…

Life on Mars worked well, partly because, I think, it had a clear two-series remit, and we knew an end would come, with all the fun of guessing what it might be and looking for signposts along the way. Ashes to Ashes neatly revives some favourite characters without the narrative problem of Sam Tyler (though is less innovative as a result, so far).

Maybe it’s time to start killing things off, and having ideas for new stories, instead of keeping the same ones going at the expense of all sense.

Posted on by Andrew Chapman

Fighting the day job

Wow. My Twitter personality test site, Twanalyst, has been used 150,000 times since I launched it just four days ago! It’s all pretty overwhelming, especially as I’m  trying to concentrate on a shedload of ordinary work at the moment… Anyway, thanks to everyone who’s used it and helped spread the word.

I’m genuinely working on new features for it, and in fact although the personality thing is a bit of fun, I think the site will have serious uses to give it longer-term appeal. For one thing, it’s useful to see stats and a user profile all on one page anyway; in future I want users to see how their stats have changed over time. I’m also working on a system to suggest relevant users for people to follow. If you have more ideas, do let me know.

Posted on by Andrew Chapman 2

The patron saint of procrastinators

A single blog post by one person I admire greatly, John Crowley, about another, Coleridge, has brought back in a rush all the haphazard things I love about the latter (and it turns out I’ve written briefly about them both before here). Crowley quotes some of this passage:

You have not above 300 volumes to write before you come to it— & as you write perhaps a volume once in ten years, you have ample Time, my dear Fellow!— Never be ashamed of scheming — you can’t think of living less than 4000 years, & that would nearly suffice for your present schemes—/To Be sure, if they go on in the same Ratio to the Performance, there is a small difficulty arises/but never mind! look at the bright side always— & die in a Dream!

The “dear fellow” he is talking to is of course himself, mocking his own tendency to scribble endless ideas for books and other projects in his notebooks – something I can relate to dearly.

Here’s another quote from the same year, 1804, collected in Anima Poetae:

This evening, and indeed all this day, I ought to have been reading and filling the margins of Malthus. I had begun and found it pleasant. Why did I neglect it? Because I ought not to have done this. The same applies to the reading and writing of letters, essays, etc. Surely this is well worth a serious analysis, that, by understanding, I may attempt to heal it. For it is a deep and wide disease in my moral nature, at once elm- and-oak-rooted. Is it love of liberty, of spontaneity, or what? These all express, but do not explain, the fact… From infancy up to manhood, under parents, schoolmasters, inspectors, etc., our pleasures and pleasant self-chosen pursuits (self-chosen because pleasant, and not originally pleasant because self-chosen) have been forcibly interrupted, and dull, unintelligible rudiments or painful tasks imposed upon us instead. Now all duty is felt as a command…

A time will come when passiveness will attain the dignity of worthy activity, when men shall be as proud within themselves of having remained in a state of deep tranquil emotion, whether in reading or in hearing or in looking, as they now are in having figured away for an hour…

His defensive stance on being allowed to dream is the tip maybe of an iceberg of agonies about his lack of output – but he still managed to outcreate most people, and much as some (far from all!) of his poetry is fantastic, I think it’s his prose which should be his lasting monument. He almost created a genre by himself – fragmentary reflections, allusions, digressions (all of which were what attracted him to Borges).

Some day I’d like to collect his thoughts of this kind into a ‘defence of dreaming’… But I probably won’t. And maybe that’s the point.

Posted on by Andrew Chapman

Thinking inside the box

Vigornian has made some good comments about the whole social media/malarkey. I feel the urge to explain why I like Twitter so much.

The biggest reason for me is a slightly odd one: I’m obsessed with the idea of ‘formal constraints’ being a spur to creativity, hence an interest in crosswords, Oulipo, J-P’s “show me the way to go home” variations, etc etc. Having to write in 140 characters or less for me is a hugely liberating idea. I don’t use Twitter to keep up with friends per se – that’s what LiveJournal and Facebook are for (among other things) – though of course it’s great to see friends there, some of whom don’t use those other sites much or at all anyway. If you like, I use it to show off to strangers.

I have reasons for that, partly: I like creating oddball web quizzes and so on, and I’ve got plans for several Twitter apps that I hope could go viral. It’s a bit depressing to put it in such terms, but much of this is about marketing – I’m not interested in the whole “drive business with Twitter” tedium that’s everywhere, but I suppose I’m ego-brand building, which might lead to interesting work (I’m self-employed, remember?), but better still just leads to meeting interesting people. So, yeah, it’s all about me – but really that’s all about encountering all the myriad creative, interesting people out there I’ve never met before. I can gain an audience for my whimsies, and be the audience for others’.

Facebook is fantastic as a shared repository for friends’ experiences (tomorrow, I’m meeting a friend I haven’t seen for 15 years – thanks to Facebook); LiveJournal is best for discursive reflection and comment – but neither helps you meet new people much. I love LiveJournal because it’s all about writing, and that’s part of what I do in life; Facebook doesn’t offer much creative expression – other than the status update, which I loved until I found Twitter – so leaves me colder.

I also love the elegant simplicity of the Twitter concept. The way you connect to others through @ and to subjects through # is very simple, but has a lot of power (I’m not saying it’s without faults, mind).

Editor says he thinks Facebook would kill Twitter by allowing public updates. maybe he’s right, maybe he’s not – but I personally prefer the sites (as Cyclotronic says) to keep their separate strengths. Trying to be all things to all people might just end up being disappointing for everyone.

I like Twitter’s search facility: I find interesting tweets and people all the time through it, all well as skimming the thoughts of a zillion people. It’s like being telepathic. Many of those are dull as ditchwater, but with just 140 characters, gems shine out. Others on the web have written better than I can on how this “live search” concept is a big thing. It’s not Facebook that should swallow Twitter – it’s Google.

I couldn’t quite get all this into 140 characters.

Small edit: I should also have mentioned how useful Twitter is for news feeds, whether national or specific (eg BBC technology) – if headline writers are any good, you can get the gist, and lately (=baby) I’ve sadly little time to read full stories. Though I still prefer RSS feeds somewhat, as the timeline gets so cluttered.

Posted on by Andrew Chapman 3

Tea for two

Gosh. Years ago I did some subbing on The Lawyer. They ran an interview with Jack Straw, where da man was very grumpy: when he ordered a lackey to bring tea, he only ordered one cup. My headline was ‘One tea, no sugar’. (I think the pic of him might even have shown the tea – I have a copy somewhere.) Cos he was ‘no sugar’, right? I had to argue for ages to get them to keep it, ‘cos they didn’t get it. Eventually it stayed.

OK, so nothing very amazing, but I was Satisfied at the time. Today, I’ve just heard from a dear friend with an absurdly good memory that he met Straw today and they were discussing headlines. He told him mine – and Straw loved it, and then told it to the editor of The Guardian.

This is probably as famous as I’ll ever be.

Posted on by Andrew Chapman 1