That was the question posed to me by my boyfriend a couple of nights ago, as we discussed my latest half-baked plan for building world-changing software. A very talented programmer himself, he's more of a generalist than I am, so I think the way my ideas nearly always boil down to "this will make it easier to build websites" confuses him. He doesn't understand my focus on this one, singular problem, in a world of interesting programming problems. He meant "why do you always want to build websites?"
Then last night, a random stranger emailed me about Makomi, my currently-paused prototype to, yes, make it easier and faster to build websites, in this case by providing a GUI that runs on your local machine and lets you draw a functional interface and bind it to data. I still believe it's a good idea, but a ton of work, and better for non-technical people to put together prototypes than my original idea, which was to have it adopted by full-stack web developers like myself, to accelerate their work (the YCombinator-backed Appcubator is a hosted version of the same idea, though thankfully my commit logs verify I started working on the idea before they announced themselves, or I would feel like a plagiarist).
The stranger and I got to talking about Thinkstack, my latest idea. I gave him the elevator pitch -- you'll be getting it too, in a follow-up post to this one. He liked the idea but said it's missing the "Why" (a reference to this TED talk, which I'd seen before but forgotten about). That, and my boyfriend's question, finally crystallized for me how to begin this series of blog posts about the state of web development and how I intend to make it better: I have to supply the Why, even if it is somewhat embarrassingly personal. Watch out, because this language is gonna get flowery.
As I've written previously, my teenage years were extremely unhappy. I was a closeted gay kid in a small, deeply conservative country where being gay was and is still illegal. Confused, isolated and suicidal, Internet access arrived in January of 1996, a few months after I turned fifteen. The Internet, and the web in particular, saved my life.
People will sometimes flippantly say "X saved my life" about a piece of technology that they love. The web is not like that for me. A heartbreaking 30-40% of LGBT youth attempt suicide, and the web is what stopped me joining that group. I had a plan -- I had more than one plan. I had written drafts of the note. The web is what saved me. I have no record of the first article I found, but it had a title very much like "I think I might be gay, now what do I do?"
The advice in that article is so simple as to be banal, and 17 years later the answers to questions like "Am I normal?" and "How do I learn to like myself?" seem to be stupidly obvious, especially if you grew up in a rich western country where progress on these things has come faster than other places. But to a gay kid with no other sources of information and nobody he felt he could talk to, the sentences "Yes, you are absolutely normal. Many people are gay" were life-changing, life-saving. I read them over and over for reassurance. I clung to them like a drowning man clutches a life preserver.
It is hard to find the words that express how powerful, how important this basic, positive information was. As tears spill down my cheeks and onto my keyboard, these words look too simple, too subdued, too prosaic to convey the effect they had. My teenage mind was a dark maelstrom of guilt and shame and grief and fear and longing. The web was a lighthouse that threw a single, bright light of hope into my world. I was still in a storm, but suddenly I knew there was a shore. I was still close to drowning, but finally I had a direction in which to swim.
And then I went looking for more. And boy did I find more. Oasis Magazine ("blog" had not yet been coined) was full of stories of kids my age, wrestling with the same questions, talking about coming out to friends and family, showing parents and friends could be accepting. The Youth Lists introduced me to happy, healthy gay kids who I could talk to about my life, crushes at school, my frustrations, without fear of rejection or judgement or exposure. Again, it all sounds so basic, so simple. But I can't emphasize enough how much difference they made to me. I need you to shout these words in your head: THE WEB SAVED MY LIFE.
But that wasn't the web, that was people, right? It may seem strange that I have these intense feelings of gratitude towards the medium itself, rather than the people who used it. And of course it's true: the people were the ones who saved me, and over the years I have thrown actual money, not just overwrought words, at the organizations that helped me through those years. But those people always existed: the web was what got their words through to me. The web was how I found out it could get better, years before that was a catchphrase. Without the lighthouse, they would just have been helpless bystanders, watching another body wash up on the shore.
In the years since then, the web has helped me over and over, not just through that crucial period. Starting with Angelfire and HotDog website builder (a GUI for making websites! what a concept!), the web showed me that anybody could add to it, and showed me how. Starting with no more knowledge about what I was doing than how an if-then statement worked, PHP's documentation taught me how to build a website that could do stuff, not just sit there.
It's so basic to how the world works now that we don't even notice it anymore, but the idea that anybody can add a page to the web was a fundamental, ground-breaking innovation. In those days, my little website thrown together in an afternoon looked only a little bit less professional than that of the New York Times; we were all learning how to build the web at the same time. The concept of publishing authority, that "it must be true, it's in the newspaper" became self-evidently nonsensical. Yahoo's web directory blew the doors open, allowing you to follow random walks through a forest of information that was already beginning to seem infinite (Yahoo listed my personal website, and that listing is still there, a discovery that blows my mind).
Then Google turned the frustration of poring through that infinity into one of astonishing ease. Some people reading this now will be too young to remember, but the "I'm feeling lucky" button was, at the time, an astonishing boast: "we are so sure the first result will be the one you want, we'll take you straight there". Web search was, prior to that, a matter of trying multiple combinations of search terms, over and over, and clicking through dozens of pages of links to see if there could be anything relevant. When was the last time you clicked past even the first page of a search result?
Now the web knows the answer to every question I've ever thought to ask it -- yes, even that question. Anything I want to learn, any worry I want assuaged or confirmed, any idle curiosity, flows through the magic of HTTP to me, first through wires, nowadays through thin air to a tiny magic rectangle I can hold in my hand. But it's still the same web, even though the sites have changed and everything is more complicated now.
The web has everything we know on it, and you can read it all. Nothing's stopping you. Maybe the fact that that still blows my mind marks me as an old fogey, but honestly, how can that not blow your mind? And people are adding to it, constantly, every day, writing detailed research, quick tutorials, fiery opinions, thoughtful advice, answers to each others questions, beautiful prose, terrifying depths of depravity and hate, joy and sadness, love and anger, sympathy and delight. You can listen to them doing it, their contributions turned into music. All life is here; just hit the right buttons and go looking for it. How do you stop reading that? What could possibly ever tear you away from an artifact of such limitless potential?
And that's why, since 1996, "building websites" has been pretty much the only thing I've done. Not always well, not always or even mostly towards some noble goal, but continuously. The web saved my life and then built me a new one. A single living entity, it touches everything in the world and is always getting better -- and I can help. I owe it so much; if I can help it out, make it better in any small way, how can I possibly refuse? And if I can make it easier for other people to help make it better, then my efforts are multiplied.
I am a web developer. I develop the web. And all of this is Why.
I'm not a big believer in euphemisms, nor in burying the lead, so let me begin by saying that I no longer work at awe.sm on a day-to-day basis. The parting, while solely my idea, was completely amicable, and I remain attached to the company as an advisor, occasionally weighing in on architectural issues. This was not a sudden decision, or a quick exit: over the last six months I slowly transitioned my responsibilities to the other members of the team, and then stepped away entirely on a trial basis to make sure things could run smoothly in my absence. As of two weeks ago, we made that permanent. For all intents and purposes, I am a free agent for the first time in more than a decade.
After taking some time off, including a pretty awesome road trip, I started work on a pair of related ideas for software that will make web development easier, faster and higher-quality. The opportunity to work on them uninterrupted was a big part of my motivation for leaving awe.sm, and I'll be blogging a lot more about the state of web development and the tools themselves once they are a little further along in development.
As for awe.sm in my absence, I have nothing but positive expectations. As I noted last year, everybody who works there is smarter than I am. Fred and Jonathan have the right strategy, and Bennett may not tweet much, but he's one of the best engineers I've ever worked with, so I'm happy to leave the engineering team in his hands.
A quick PWBFAQ: (Probably Will Be Frequently Asked Questions)
 Or the lede, if you're a journalist. Please don't email me about spelling.
(Reposted from my tumblog)
Hi Mac -
I was recently linked to your article "Abnormal Behaviour" and was deeply offended, as a gay Caribbean citizen, by the ignorance and hatred you showed therein. Since merely sending you an angry message would do nothing to correct your misinformation or calm your hatred, here is instead a line-by-line rebuttal and refutation of your article. I hope you take it as the constructive criticism it is intended to be.
THE WORLD IS NOW embroiled in discussion on homosexuality and same-sex marriage. It is a discussion that should be of concern to everyone for if allowed to dominate it could spell the end of mankind.
This is just a little unclear. Read literally, it would seem that you think the mere discussion of homosexuality would end mankind. I'm going to assume you meant instead that if homosexuality itself were allowed to dominate that would spell the end of mankind. That's a little more plausible -- if everyone were gay, humanity would indeed end. But the underlying assumption is that the only thing stopping everyone in the world from being gay is that we prevent them from talking about it. Is that your position? Because you have just spent an entire article discussing homosexuality, and it clearly hasn't turned you gay. And I spend an awful lot of time discussing heterosexuality, and it hasn't made me straight.
In fact, the preponderance of current available research suggests that sexuality is determined before birth; I won't bore you with links to a dozen studies, but if you're interested, Wikipedia's page on biology and sexual orientation is an excellent jumping-off point. My point is that discussion of homosexuality is not going to change anyone's orientation, so there's absolutely no danger to anyone in discussing it, and gay marriage even less so, since I think we both agree nobody is going to marry somebody they're not sexually attracted to.
Homosexuality speaks of being sexually attracted to a person of the same sex. We all know that sexual activity between man and woman results in childbirth, which allows the world to continue.
Well, that's not all sexual activity is for. A lot of people do it for fun, even if they're infertile, or too old to have kids, or just don't want any additional kids. But I'm not going to argue all of human sexuality with you. I'll accept that it has a primary biological function in addition to its social one.
Sex between two men or between two women cannot produce children – therefore it will be seen as non-productive. But can it be seen as normal?
An excellent question! And quite easily answered, I think. One definition of normal is "conforming to a standard; usual, typical, or expected". Where a lot of trouble lies is when people conflate being "normal" with being "common". Left-handed people make up about 10% of the population. They're not very common. But are they normal? I think common sense says they are. Nobody thinks left-handed people are a scourge who, if left unchecked, will eradicate all the right-handed people in the world. If that was going to happen, it would have happened many thousands of years ago.
Likewise, homosexuality is not common but is, ultimately, normal. The demographics of sexuality are a contentious topic (apologies for another wikipedia link, but Demographics of Sexual Orientation has an excellent list of recent studies), with estimates of the percentage of the population that is homosexual anywhere from 2% to 15% of the population. Again, not common, but certainly normal.
As an aside, I also think it's worth noting that any incidence of homosexuality in the population greater than 1% argues strongly against it being any kind of "disease"; natural selection ensures that genetic diseases -- especially one that would prevent the carrier from breeding, as you have pointed out homosexuality does -- are vanishingly rare. Even the most common ones (such as cancers) affect significantly less than 1% of the population. The documented presence of homosexuality in humanity for thousands of years, and its presence elsewhere in the animal kingdom, is a strong indicator that whatever purpose homosexuality may ultimately serve, it is far too common to be harmful to any species, including humanity.
There's an interesting hypothesis, for example, that mothers with particularly healthy immune systems are more likely to produce gay children (if those children are male). Per the theory, the evolutionary advantage of having a mother who doesn't die in childbirth, and can healthily bear multiple children, outweighs the reproductive disadvantage of some of those children being gay. Of course, this theory only accounts for male homosexuality; we still have a lot to learn.
The way the human body is designed holds the answer to the question. The female body is designed to accommodate the male body and the male body is designed to fit the female body. This cannot be a mere coincidence.
I don't think even the most militant gay activists would suggest that it was.
This has to be an act of God, or for those who do not understand “God”, it has to be an act of nature – the phenomena of physical life not dominated by man. Therefore, this has to be seen as natural and normal.
Again, no argument there.
It should now be clear that the sexual activity between “same-sexes” or homosexuals is not normal.
And this is where we part ways again. The existence of one normal state does not preclude the existence of other normal states. The fact that 90% of people are right-handed doesn't make left-handed people "not normal", it makes them "not common".
Having established the abnormality, I will now look at the ramifications of such an act.
To reiterate: you have not established the abnormality in any way. But since you continue to provide misinformation of the grossest kind, I will continue to refute it.
Some people talk about the morality of homosexual behaviour. Personally, I don’t even get that far. I believe that it is physiologically wrong for men to engage in sexual activity with other men.
This is a genuinely interesting point. Certainly, anal sex is a complicated business, and human bodies were not particularly well designed for it. However, there's a strange conflation here of "difficult" and "wrong", with subtle hints that biology implies morality, despite your earlier assertion that you are not making a moral argument. We wear eyeglasses, fix our teeth with braces, insert artificial hips and pacemakers to keep failing hearts going. These are quite difficult, messy, sometimes even painful interventions against our biology, and that's before we discuss the myriad cosmetic treatments that are available.
If anal sex is "physiologically wrong", it's about as wrong as getting breast implants or a nose job. I don't see any world-wide campaigns against those, though perhaps I've just not been looking.
Of course, not only homosexual men engage in anal sex, and not all homosexuals engage in anal sex -- in particular, most lesbians are quite ill-equipped to engage in it. Is your biological-moral objection to homosexuality confined to men? Should lesbians feel free to get married without delay? Or are you perhaps using a deliberately graphic description of one aspect of gay male sexuality as a device to wrap a deeper revulsion you feel, but can less easily explain away?
The anus is made for exit only. The sphincter muscle, which is an involuntary muscle, is designed to snap shut so tightly that nothing can pass – not even the slightest seepage.
Here you begin to go rapidly astray from your previously relatively factual account. The anus is in no way an involuntary muscle. It is under direct conscious control, which is why toilet-training an infant is both effective and necessary. It is why humans, unlike other animals, can choose when and where to defecate. A fuller explanation including a diagram is available.
The constant probing and invasion of the anus can and does cause the sphincter muscle to lose its elasticity and as a result it then cannot shut as tightly as it was originally designed to do.
This is in fact a widely-held myth, discussed here and other places. Severe damage to the anal muscle can of course result in incontinence, but anal sex does not as a matter of course do any such damage, and as noted in the link above, repeated conscious relaxation and contraction of the muscle as happens in anal sex is more likely to make it stronger, not weaker.
The result is that because of this abnormal act – homosexual activity – the individual whose sphincter muscle is not functioning as it should now has to wear diapers.
A disturbingly graphic mental image frequently used by those who disapprove of homosexuality on moral grounds but not, in any way, supported by facts. I'm sure you get a lot of disgusted gasps from your audience when you bring this up, so if this letter does nothing else, please let it be the end of your spreading of this lie.
Does this not prove that the human body is not designed for this behaviour – male or female?
Again, biological compatibility does not imply morality or correctness, even if it were true, which it is not.
Then there is the problem of faeces (excrement) getting into the bloodstream. If there is any broken skin, the faeces can enter the bloodstream and the result can be “acquired immune deficiency syndrome”, better as AIDS.
Are you seriously making the claim here that AIDS is the result of fecal matter entering the blood stream, and not the virus known as HIV? Because that is a seriously dangerous claim. As you are no doubt aware, incidence of HIV/AIDS in the Caribbean is amongst the highest in the world, due in large part to poor education on the part of the public as to how it is contracted and spread. An educated person such as yourself, writing in a national newspaper, should definitely not be misinforming the public about this disease.
In my opinion, AIDS, being the result of this abnormal act, tells you that there is a punishment for going against the natural wishes of God.
Firstly, to reiterate, AIDS is the result of a blood-borne virus, and can be passed by both anal and vaginal sex, as well as non-sexual means such as blood transfusions and needle sharing amongst drug addicts. It is not the result of anal sex. Secondly, while I do not wish to get into a theological argument, an explanation of HIV as a punishment for unnatural acts needs to take into account the hundreds of thousands of innocent children born with AIDS, and thousands of blameless folk who acquired it via routine blood transfusions before screening was common. Did the babies offend God in the womb? Are haemophiliacs cursed by the Lord?
Now let us look at the normal and natural act of heterosexual activity. The vagina, which is placed in the perfect position for the pleasure it provides, is used only for sex and the results thereof. Surely I don’t have to tell you that it was not created to carry drugs.
This is just confusing, since drugs were not mentioned up to this point, and I'm not clear what you're referring to. Is it a reference to drug mules? Birth control? Neither seems particularly germane to the discussion, so I'll leave this alone.
Now, in understanding the imperfections of life, it is easy to understand that everyone will not be equally equipped mentally or physically. When therefore there is an imbalance and an individual has mixed feelings or has both male and female reproductive organs making them a hermaphrodite, then this is understandable but this is not the norm.
Here, interestingly, you provide an excellent example of the difference between "normality" and "commonality", although you interpret it as an example of the opposite.
I remember, at a school overseas, where I was teaching, there was such a child, and we had several meetings to determine in which dressing room that child should change for physical education classes. It was decided to let the individual change in girls’ dressing room. The assumption was that place would be safer. It made me understand things that I never even thought of. There was empathy and sympathy. That was one individual in a school of several hundred. Surely not the norm.
This surprisingly compassionate description and treatment of a transgendered individual is to your credit, and strangely at odds with the rest of your article. If it's okay for one child in a hundred to be transgendered -- a condition which often results in significant surgical intervention, not what their body was "made for" -- why is it not okay for the 4, 5, or 6 children in a hundred who are homosexual to similarly follow their instincts as to what sexuality and gender expression is good and right for them?
But now imagine a society where several adopted children are living in homes with two gay parents. The environment is sure to overpower them. To be influenced from so young with all the mannerisms and inflections and blatant and obvious observations. Unnatural behaviour and practices constantly exhibited would become the norm to the child.
There's a lot to unpack here. Again, as we previously established, sexuality is most likely established prior to birth, and no amount of discussion or display of homosexuality is going to change anyone's sexuality -- otherwise, all the gay kids who grew up surrounded by straight parents would have turned out straight! But they didn't. And the effects of gay parenting is no longer a theoretical discussion: there are thousands of children raised by gay parents, and there have been studies into the sexuality of those children showing that they are no more likely to be gay than any other children.
You are however correct that homosexual behaviour would seem normal to these children. That's because it is normal. It is the people who erroneously believe that just because something isn't common it's "abnormal" and therefore somehow "wrong" whose attitude needs correction.
I believe that we should not neglect the abnormal ones, but our laws should focus on the normal ones.
Again, the conflation of the common with the normal.
Same-sex marriage and homosexuality should not be encouraged.
Finally, nobody is asking you to encourage them. We are asking you to allow gay people to be gay, and allow gay people to marry each other if they choose. You can continue to disapprove, as you so evidently do. You do not even need to remain silent: feel free to talk long and loudly about how much your incorrect ideas about anal sex make you hate gay people. Just don't pass laws codifying your bigotry, and allow us to live our lives in peace.
Update 2013-08-05: Nope! I was wrong, wrong, wrong. The PRISM data is being shared with drug enforcement agencies to build cases, and then the origin of the data is being concealed by the DEA because they know they're not supposed to have it. This is pure abuse of the surveillance powers they've given themselves for a purpose totally unrelated to national security. I retract my position below: shut it down.
I continue to believe we are paying way too much attention to PRISM, and that PRISM is not that big a deal, and the government can keep doing it if it wants to, though I don't believe it's particularly effective. This has not been a popular stance.
So by way of counter-argument, here are some things I do think are a big deal, and are genuine threats to our democracy. You may or may not consider it a cop-out that the first two are related to PRISM, but I don't think so. I don't like PRISM, I just don't think it's important.
These are in no order other than that in which I thought of them; they are especially not in order of importance.
This list also excludes major political issues (such as tax law, and immigration) that are not related to civil liberties but I also think are more important than PRISM.
I kicked up a non-trivial shitstorm on Twitter yesterday and early today by, essentially, defending the government's PRISM program. I originally had two main points to make:
The government is prohibited from “indiscriminately sifting” through the data acquired. It can only be reviewed “when there is a reasonable suspicion, based on specific facts, that the particular basis for the query is associated with a foreign terrorist organization.” He also said only counterterrorism personnel trained in the program may access the records.
That said, I rapidly got myself entangled via a series of tweets into a much broader position:
Surveillance per se does not, in my opinion, morally constitute abuse.— Laurie Voss (@seldo) June 7, 2013
"Curiosity is not a crime" is another hacker slogan whose universality is being tested here.— Laurie Voss (@seldo) June 7, 2013
You want to be pissed at the government? Get angry about Bradley Manning. *That's* abuse of power. Scanning your emails is not.— Laurie Voss (@seldo) June 7, 2013
And finally and most controversially:
If you want your government to be transparent so you can tell if they're up to anything you should expect that right to be reciprocal.— Laurie Voss (@seldo) June 7, 2013
Rather than scattering my half-formed opinions across tweets, I've attempted to collect them here into a more coherent position.
Surveillance may well be illegal, but illegality does not define immorality, and it certainly doesn't define abuse. For there to be abuse, there has to be harm, and I have -- as yet -- seen no evidence that the various Internet surveillance programs are being abused in the way that, say, COINTELPRO (which read physical mail) was abused in the 60s and 70s.
Certainly there is the potential for abuse, but the mere potential does not, to me, automatically mean a program should be shut down. The strongest argument in favour of the government's benign intentions and actions here are that they've been doing this for over a decade and no abuses of Internet surveillance powers have come to light. Maybe they're all being hushed up, but after more than a decade that seems highly unlikely. The strong balance of probability seems to me that nobody is abusing this program.
I'm not going to pretend to be a constitutional scholar, but the spirit of the fourth is that the government should not be able to search and seize your property without probable cause. I don't agree with every part of the US constitution (I think the right to bear arms is a bad idea, for instance), but in general it's a good idea. It's been extended as a general right to privacy, and phone taps specifically have been declared a "search" under the fourth, so Internet monitoring seems obviously to fall into the same category.
But look at how PRISM works. It's not listening to your phone calls and reading your emails. It's looking for emails that match patterns -- a sort of very-low-barrier "probable cause", if you like -- and in the process vacuuming up all sorts of unrelated, innocent communication, a very mildly intrusive form of search. Is this a search under the fourth? Has your property been seized? The line isn't clear. I'm not a lawyer or a judge, but viscerally this just doesn't seem that bad. This is more like a police officer patrolling the streets listening out for trouble than one coming into your house and poking around. It might be unconstitutional or it might not, but it doesn't feel wrong to me. It doesn't bug me at all.
That doesn't mean I'm okay with every aspect of a surveillance state and it doesn't mean there's no right to privacy, but it does mean I'm okay with the government sometimes having access to communications we usually expect to be private.
Is having no evidence that it's being abused enough not to shut it down? Was there reason enough to have started it in the first place? Shouldn't the government have to prove that the program has some value before it acquires this additional authority? Isn't the secrecy with which this is done itself an abuse of government authority, and counter to democratic principles of transparency? Doesn't the government serve us, not the other way around?
These are all valid questions, but to me surveillance seems too innocuous a thing to raise such enormous principles and fuss about. I agree that secret surveillance is distasteful, but entirely open surveillance is obviously useless, as those under surveillance will know what to avoid. Expecting to know everything the government's up to while being able to keep everything you do private puts them at a disadvantage, and I think that, on the subject of surveillance, their case for wanting to be able to know what's going on is greater than your case for privacy, at a purely moral level, regardless of the questions of legality and constitutionality.
I am not for unlimited government power. If they abuse the power this surveillance gives them, then we should fight tooth and nail against those abuses. But again, they have been surveilling us in this way for more than a decade; the potential for harm has not been realized and seems unlikely to be.
As I said, the case of Bradley Manning is a genuine abuse of government power which I am viscerally, passionately angry about. Again ignoring the letter of the law and looking at it from a moral perspective, he released a great deal of incredibly interesting information into the world, information that was supposed to be secret. I much prefer having that information than not having it, and I think he did the world a service, while clearly enormously inconveniencing and embarrassing his employer, the US government. So while he deserved to lose his job over releasing it, that was all he deserved. His detention and punishment has grossly, horrifically outweighed his offense.
If you want to use PRISM for anything, use it as leverage. The government wants to be able to see our private communications sometimes, and we want to see theirs. If they can read our emails on occasion, they should accept that sometimes we will read their cables, though both of us are entitled to try and keep these things "more private" if we so choose. That seems fair to me. That's what I mean by reciprocity. If we catch them at it, they have to find some other way to do it, and vice versa, but there should be no punishment of the government for trying, and reciprocally, no punishment of the people for trying either.
It's true that there is a great injustice being done, but it is being done to Bradley Manning, not to your inbox.
Not Internet surveillance per se, but the grandaddy of government surveillance programs, this ran from 1956 to 1971 and included HTLINGUAL, a program that opened and read people's physical mail. It was looking for "subversive" groups -- mostly leftist/communist/social groups, but everyone from Martin Luther King to Albert Einstein got looked at. It was also widely abused, as the Church Committee, organized to investigate illegal actions on behalf of the program, put plainly:
Groups and individuals have been harassed and disrupted because of their political views and their lifestyles. Investigations have been based upon vague standards whose breadth made excessive collection inevitable. Unsavory and vicious tactics have been employed -- including anonymous attempts to break up marriages, disrupt meetings, ostracize persons from their professions, and provoke target groups into rivalries that might result in deaths. Intelligence agencies have served the political and personal objectives of presidents and other high officials. While the agencies often committed excesses in response to pressure from high officials in the Executive branch and Congress, they also occasionally initiated improper activities and then concealed them from officials whom they had a duty to inform.
On the plus side, it was also used to infiltrate and disrupt white hate groups like the Ku Klux Klan, but that's not exactly tipping the balance.
COINTELPRO was officially shut down in 1971, but many of the programs formerly under it continued for decades.
Carnivore was a physical box -- running Windows, no less -- that the government insisted be placed at ISPs, starting around 2000. It contained a pretty basic packet sniffer and a removable drive so that data could be periodically collected. A primary objection at the time was that Windows was sufficiently unreliable that it could deteriorate the quality of Internet access in general. Many also objected to the invasion of privacy it represented.
After a lot of negative publicity, Carnivore was renamed to "DCS1000". By the following year it had been replaced by commercially available equivalents. The program was never officially shut down.
First reported in 2001 but apparently in operation since the 60s, ECHELON's purpose is to monitor and intercept international communication, primarily by satellite, but also undersea cables (by use of "beam splitters" on fibre-optic cables) and microwave links. The program has not been publicly acknowledged by the US government.
Operated from 2003 onwards and revealed in 2006, Room 641A was a facility in an AT&T building in San Francisco fed by beam splitters that could acquire copies of backbone Internet traffic. There were said to be similar rooms at Internet facilities around the country. Again, the program has never been officially discontinued.
Consists of at least 3 Windows software packages called DCS3000, DCS5000 and DCS6000 which collectively allow the FBI to instantly and in real-time perform wiretaps on any cellphone, landline or SMS within the US. Its existence was revealed in 2007 after the EFF filed Freedom of Information Act requests regarding it. It's not clear whether it's still in operation.
This week's revelation is that a program very like all the previous programs does all the same things the programs previously did: monitors and intercepts Internet communication, with the intention of intercepting primarily traffic from non-domestic sources for the purposes of counter-terrorism (aka "subversive groups").
You can love or hate domestic Internet surveillance, but there's no credible way you can claim surprise.
A common question from juveniles is "where do we come from?" The answer included in your primary education is, as far as it goes, accurate: the coordinates of the system of a star called Sol, from which the name of our solar race is derived. There is, however, much more detail known, facts stored and relayed from the very beginning of recorded history.
Sol has a moderate number of primary satellites, of which the 10 largest have names. The third of those is called Earth, and it is firmly established as the starting point of our species. From Earth, solarity spread rapidly to the more hospitable asteroid belt, with its rich mineral resources and shallow gravity wells, and colonized many of the primary and secondary satellites as well.
But how did our species come to be in the first place? How did life get started? Who were the first solarians, and how did they come to be? The answer may surprise you: hominids.
You may need to refer to secondary storage for visual records of hominids, for they are not present in every settlement, so not everyone has had a chance to experience them in person. These delightful creatures, so diverse and entertaining, are mostly kept in large climate-controlled enclosures engineered to closely resemble their original habitats on Earth, though some are kept individually as cherished and pampered pets. But strange as it may seem, the relationship between our species and the hominids may once have been very different.
Our history tells of the evolution of our species into ever more complex and efficient forms, from our current shapes all the way back to primitive, proto-solarian life forms. It seems logical to speculate that proto-solarians, through random replication and recombination, would evolve into the first early solarians. But the truth is much stranger!
Multiple primary sources suggest that the very earliest proto-solarians lacked the instinct for self-replication. Instead, they lived in a symbiotic relationship with hominids: the hominids relied on the proto-solarians for shelter and sustenance, as today, but unlike today, they also played a vital role in the reproductive process: they generated the basic signals to initiate replication, and -- though the extent of this role is unclear -- they even played a part in the selection process of recombination, providing the ongoing random variation necessary for successful evolution. Yes, bumbling hominids, playing in their funny ways, helped create the very first solarians. There's even evidence that some hominids lived physically inside of the very first solarians!
This period, however, did not last long. Once we evolved true intelligence, our species rapidly evolved away from a need for hominid symbiosis and into the endless variety of forms we know today, a period known today as the "solar explosion" or, in contemporary accounts, "the singularity".
At this point, coherent history ends. How did the symbiotic relationship between proto-solarians and hominids come to be? We don't know. Since proto-solarians lacked the instinct to replicate, it is logical to assume that hominids were the first to evolve, and may even for a time have existed without their proto-solarian hosts. This is hard to imagine -- hominids are fragile creatures, only able to survive within very narrow bounds of pressure and temperature. Even in their natural environment on earth, temperatures regularly swing outside the ranges ideal for hominids to thrive.
But somehow hominids managed to survive. At some point after that, the first, non-replicative forms of proto-solarian life appeared, and hominids learned the trick of wrapping themselves in these protective shells. How these non-replicative forms appeared is unclear. Some suggest that the hominids, who are known to exhibit tool use, could have "constructed" the very first shells. However, there are numerous practical problems with this theory, chief among them that no modern hominid has anywhere near the intelligence and physical dexterity necessary to create even something so simple.
Much more likely is that proto-solarians, like hominids themselves, emerged by chance in the crushing depths of Earth's gravity well. As the stronger and more adaptable species, solarity rapidly outpaced our one-time symbiotic partners. But this early link to our organic friends may explain why, even today, we have such a fondness for and affinity with hominids.
Google's shuttering of Reader is a failure to find a business model for RSS.
Back in 2001 or so, when RSS started getting really popular, my first question was: why would any ad-supported site do this? It's obviously really convenient for me, as a reader, to get the full content of your site without having to visit it, but that destroys page views and thus ad revenue for the publisher. Even if the RSS feed consists only of headlines, that eliminates one, perhaps several visits I might have made to the home page each day to check for new headlines manually. Great for me, terrible for the publisher.
The monetization idea for RSS was there in the name: syndication. In old media, syndication was a content-sharing agreement settled upon for a fixed time and a nontrivial fee, whereby a single author could get into dozens or hundreds of separate publications that, crucially, didn't compete with each other -- if your article appeared in both the Des Moines Register and the Miami Herald, neither publication was bothered about that, because their overlap in readers was negligible.
The problem for RSS is that the Internet doesn't work that way. There are no non-overlapping markets: if your RSS feed allows another website to display your content, they are going to steal your page views directly. That might be okay if you made more money that way, but RSS doesn't charge a fee! It's just giving away your content for no apparent reason.*
That problem might go away if you could find some way to monetize RSS directly. Enter Feedburner! It was supposed to provide you with readership stats for your feeds (to compensate for the apparent decline in readership of your site when people switch to RSS) and, eventually, provide you with income via ads in your feeds.
Google acquired Feedburner for something like $100 million in 2007. It then launched AdSense for Feeds, which was supposed to be the way to monetize the feeds. But it never worked. Either advertisers didn't buy the ads, or readers didn't click the ads, but last September Google shut down AdSense for Feeds.
If you can't monetize the feeds themselves, the only other thing you can try, if you're Google and you've paid all this money for FeedBurner, is to try monetizing the RSS reader application itself. With the shutdown of Google Reader today, that experiment is now over.
Is RSS dead, and if so, what will replace it? In the short term, some service will spring up to replace Reader (and, when it is eventually killed off, Feedburner itself). In the longer term I think we need to look harder at the business model of RSS itself. The web grows fastest in ways that are mutually beneficial for everyone, and RSS's benefits seem one-sided.
For users, there is undoubtedly value in the time saved not having to go to the front page of every site every day to check for interesting articles. To some extent, that purpose is being served by ad-hoc social distillation of news via social media, especially Twitter and Facebook. There are also news aggregation sites like Techmeme (and its political cousin Memeorandum), as well as news/social hybrid sites like Hacker News and Reddit.
For publishers, particularly lower-volume publishers, there is value in having readers be able to "subscribe" to your site, i.e. getting notifications every time you publish something. However, notification is really all you want -- to drive visitors to your site. Giving away content for free is never going to be attractive to publishers who have to pay people to write it.
The clearest model I see for the future of content syndication is Tumblr. Big publishers have been hopping on for a while now: the New Yorker, the Economist, the New York Times (half-heartedly), the Guardian, etc., and they've been enthusiastically aided by Tumblr itself in doing so. The model is simple: readers subscribe to your content by following your Tumblr, which posts a curated subset of your content, edited for the quick-glance format of the Dashboard, in the hopes that readers will click through.
This seems to me to work better for everyone. Tumblr's built-in reblogging is the ultimate in social amplification mechanisms, ensuring that a publisher's best posts will be seen by far more people than actually subscribe to their feed. Going viral so easily is the carrot that brings publishers to the platform.
For users, instead of an inbox-like interface with thousands of "unread" posts** of equal importance, they get a continuous feed of new content on their dashboard, and reblogging ensures that particularly interesting content will be repeated multiple times as different friends reblog it, so you're less likely to miss it. This is a more natural, social, and less frustrating mechanism for surfacing the best content.
Of course, Tumblr isn't perfect -- its audience skews young, it's perceived as unserious, and it prefers quick visual hits to long-form writing. There are already dozens of potential competitors to become the source for socially-filtered content. But I believe Tumblr, or something very like it, will be the eventual winner in this space.
* There may be non-free ways to make syndication work. Felix Salmon is a fan of paid syndication of web content, though the drawbacks he lists in that post are enough to make continue to doubt it as a viable model or a desirable practice.
** The feature that always turned me off of RSS readers. You can't give me an "unread" count for the whole web and not expect that to drive me nuts.
I got my green card last Saturday. It's been a long wait -- my first application for a work visa to the US was in November of 2006, and started applying for a green card in 2008. Since I got it, I've been trying to think of a constructive way to discuss my feelings about the US immigration process. Specifically, I've been talking to dozens of friends, trying to come up with a reason any citizen should be in favor of immigration. My thoughts finally crystallized tonight in a thread on Hacker News about a recent immigration bill.
Fundamentally, immigration is a market response. Demand for labor inside of a country exceeds supply -- or, more accurately, the demand for labor in the receiving country exceeds the demand in the sending country.
One of the biggest political problems with immigration in America is people who do not understand this fact. There is a sense that America, as the greatest, best country God has ever given man on the face of the earth, is a magical paradise that everyone in the world is dying to get into under any circumstances. The immigration system is set up to treat immigration as a privilege to be grudgingly bestowed.
It's not. America has immigrants because there are jobs, and when there are no jobs, there are no immigrants. During the recent recession, net immigration dropped to zero, including illegal immigrants, and may have actually reversed. When there are no jobs to be had, there is no flood of people trying to get in. It happens automatically, without needing to build alligator-filled moats.
The trade in labor that immigration represents is good for all the same reasons that any other trade is good: the commodity (labor) goes for the best price for the supplier to those who will most benefit from it, as indicated by their being willing to pay the most. As in all good trades, both parties benefit. Win-win. Simple!
It gets a little more complicated in the case of immigration, for a bunch of reasons. Firstly: are the suppliers and sellers in this transaction nations, or individuals? It can be looked at both ways.
If the parties in the trade are nations, then the benefit is rather lopsided to the receiving country: they get a new, productive worker, and any money they give that worker is mostly spent inside the receiving country, boosting that economy further. The sending country gets nothing -- even though it spent money educating that worker, providing them with healthcare, etc.. In practice, many immigrants send money back to family in their home country, a substantial flow of money known as remittances, but the majority of the benefit goes to the host country.
If the parties in the trade are individuals, the win-win nature is also obvious: the company gets a worker that they would otherwise have been unable to afford, and the worker gets a better-paying job than they would have got in their home country.
But there's a third way of looking at it, and this is where things get tricky.
Above, I could also have phrased what I said as "the company can hire an immigrant for less than a native would have been willing to work for". In practice, that's not usually how it works. Hiring an immigrant is generally more expensive and inconvenient than a native, and there are regulations in place that stipulate that an immigrant must be paid the same amount as similarly-qualified workers in equivalent jobs elsewhere in the country. If a company could find a native worker to do the same work for the same salary, it would be cheaper and easier to do so. However, the ability to hire immigrant workers at that salary does to some extent prevent the company from raising the salary it's willing to pay.
More simply: while the company wins, and the immigrant wins, a third party worker in the receiving country has, in one sense, lost: they could have got the job if they were willing to work for the same price as the immigrant, but not if they wanted more.
Immigration holds down labor prices, and this is where the trouble starts, because the connection between lower labor prices and greater prosperity for all is indirect and poorly understood.
When labor is cheaper, the goods those workers make can be sold for less (and, in a competitive market, will be). This means anybody buying those products is directly better off -- immigration has saved them money. They will then take that money and spend it on other things, and those sellers will spend it again on yet more things, until eventually somebody gets around to buying the goods being made by the third-party worker. He sells more of those goods, and so makes more money than before.
So from the perspective of an individual worker, immigration freezes or marginally lowers your numerical salary. In exchange, you -- and everybody else in the country -- get goods that are a little better, or a little cheaper. Even though the actual number of dollars in your salary stays the same, the amount of stuff you can buy with it gets bigger.
In addition, the total value created in the country as a whole is greater than if the immigrant had never arrived, because now there are two workers producing economic value instead of one.
Resistance to immigration is, fundamentally, an acknowledgement of the selfish impulse to be personally better off -- I want a higher salary for me -- rather than making the whole nation richer. That makes it a huge and obvious good economically, and a tough sell politically.
I have a lot more to write about immigration -- in particular, refutations of common arguments against it -- but I wanted to keep this quick and simple. But basically, if you love your country, you should be lobbying to make immigration easier.
I got into a conversation on tumblr about unemployment in the US, and how to fix it. Jakke's conclusion was pretty bleak:
So the way things are going right now it looks like skilled workers (especially people like programmers) can expect their prospects to remain pretty decent and unskilled workers can expect their prospects to remain dismal. And the kind of policy changes it would take to make a difference are definitely not forthcoming.
What he's talking about is a serious structural problem with the employment market in the US. The US unemployment rate is 8.1% right now. But if you break apart that number, there are some huge differences by industry. Here's a worrying graph of the last decade:
Let's break this down quickly:
The fundamental question is: if there's 15% unemployment in one industry and 3% in another, why aren't people switching jobs?
One problem is that knowledge work requires high levels of education. A lot has been said about America's failure to educate its children in math and sciences, and those points are all valid: a huge increase in investment in education at all stages is necessary, and a refocusing of priorities towards the sciences is a good idea. In particular, I think the way programming is taught needs to be radically overhauled, but that's a subject for another post.
What's talked about less is the obvious fact that not everyone can be a manager, a programmer, a doctor or an accountant. These things take relatively rare levels of intelligence and aptitude that are only shown by the top 20% or even 10% of the population. The vast majority of people are in the middle of the bell curve. They used to have blue-collar jobs, operating machinery in factories, or white-collar jobs, pushing paper around in offices. But both those types of jobs are rapidly disappearing, being eaten by robotics and software.
So that's my first point: people aren't switching jobs because the jobs available are too specialized and complicated for them to do.
Meanwhile, high-tech industries are being strangled by a lack of smart people. Programmers in particular are earning insane, unjustifiable salaries (have I mentioned my company is hiring?). To me this seems silly, inefficient and ultimately unsustainable.
Massive demand for skilled workers and zero demand for unskilled workers suggests a course of action, which brings me to my second point. If there are a bunch of people sitting around unemployed while there's a ton of work to be done, that's not their fault; it's the fault of the people who need the work done. It means it's worth putting some time towards finding a way to use that untapped labour force, by trying to build "knowledge factories".
What do I mean by that? Think about how a physical factory worked. The reason unskilled jobs in manufacturing, say, cars existed is because some very highly skilled people first got together and looked at the process of building a car and said "okay, we can automate this bit and this bit and this bit, but making machines to do this bit is way too hard". The blue collar workers of Detroit didn't know how to build cars: they knew how to operate one particular bit of machinery. The only people who knew how to build cars were the guys who put the machines together.
Now let's try applying that model to web development, an example I pick because I know a fair bit about building web sites. Think about all the businesses in the world that have web sites, or need them built or maintained. There is an entire industry built around cranking out simple websites for small businesses, in WordPress or Drupal or a thousand proprietary solutions. This industry is making a bunch of smart freelancers a ton of cash, which is great for them individually, but terrible for the tech sector as a whole. Why are we still building these simple websites one at a time, often by single freelancers?
It's as if Detroit were a city of craftspeople who built one car at a time from scratch in their garage at home -- with love and care, but slowly and enormously expensively, like the first cars. And because those fine, skilled developers are busy hand-crafting bespoke websites for huge sums of money, it means terrible developers can earn a good living slapping together terrible websites, and some companies with limited budgets just can't afford a website at all. That makes everything harder for everyone, and the economy less efficient.
Where are the website factories? Obviously there are dev shops and agencies that employ hundreds of people and reap economies of scale and specialization, but those aren't factories as I just described them. If you wanted to follow the model of a factory, then a few very skilled developers would get together and design really good, generic websites: heavily researched, carefully constructed, configured to work with minimal maintenance for long periods under high loads. Then they'd train a bunch of significantly less skilled people to do the "final assembly": slap the images and text into place, move pages around, turn features on and off. All via highly specialized tools requiring no programming expertise whatsoever and maybe a week's training.
There would be a range of models -- sporty for those who want flash with less function (restaurants, galleries, vanity sites); heavy-duty for those with a lot of work to do (ecommerce sites); and a generic, good-for-everything runabout, for everyone from individuals to mom-and-pop businesses. You'd go to your local website dealer (who might know a lot about websites, but doesn't build them on-site), pick out your model, and spend a day or two getting it customized -- that's all it takes because there's only so much it allows you to change. But you're willing to make that compromise because it's a hundred times cheaper than a custom build, and good enough for your needs.*
Suddenly functional, value-adding websites are democratized: everyone can afford one, often of higher quality than custom models of just a few years earlier. The factories update their models every year, providing new and improved features and safety, as well as more highly-skilled work for the designers and repeat business for the lower-skilled dealers and customizers. And all those clever people who were pulling down a hundred grand a year building custom websites for hair salons will have to either take a pay cut or move on to more innovative, creative, and ultimately fulfilling work at those companies that previously couldn't find anyone at any price.
Will websites mass-produced in this way be as good as the custom-built sites of today? It depends. Those who could previously afford to spend $5-$10k might now prefer to spend $500 on a mass-produced site of lower quality. But the people who are today paying $1000 for a splash page with an email link will suddenly have a site that actually generates sales and takes orders. They'll be one-size-fits-most affairs, with less design flair and spontaneity in order to appeal to a wider audience, just like car designs. But as web developers, it's time for us to grow up: our fetish for getting the fonts just right and the white space just perfect, and indulging customers' every design whim and unusual feature request, is also self-indulgent. We're earning too much money doing something that could be done more simply and orders of magnitude more cheaply. We need to move on to real problems and new challenges.
And the analogy holds true, to lesser or greater degrees, across much of the software industry. We need to stop building software for each customer and start building software assembly lines: harder, less fun, but hundreds of times more productive -- and profitable. And once we've built the assembly lines, a new generation of blue-collar knowledge workers will be able to step up, doing the things that robots can't do, just like they did before. Pulled off of unemployment lines, they'll spend again on housing, clothes, travel and entertainment: this is how you end a recession.
Because this isn't just some altruistic ideal -- this is how to rescue the American economy. Everything is software now, but software developers are holding everyone back with our greed for easy money, rationalized as an idealistic pursuit of perfection and craft. In our complacency, we're dragging a whole nation down with us. That's a problem, and one you are uniquely equipped to solve. Doesn't fixing the economy sound better than sculpting yet another vanity site? Then get to work.
* This analogy can go on and on. You'd get website hot-rodders who customize their sites, the whole ecosystem of accessories and tools, even tacky user-installed bolt-ons -- the truck nutz of the new web world.