Expressing Data Dependencies in React

One-way data flow is a foundational concept in React (See? It’s on the homepage!). React is very opinionated about how data should flow…

Component Style

I think styling is one of the most interesting parts of the frontend stack lately.

Learning to Code

Yes Yes Yes, to everything in this incredible piece my friend Anna wrote about programmer and writer Ellen Ullman: Hacking Technology’s Boys’ Club:

“I am dedicated to changing the clubhouse,” Ullman told me. “The way to do it, I think very strongly, is for the general public to learn to code.” It doesn’t have to be a vocation, she said. But everyone, she argued, should know the concepts—the ways programmers think. “They will understand that programs are written by people with particular values—those in the clubhouse—and, since programs are human products, the values inherent in code can be changed.” It’s coding as populism: Self-education as a way to shift power in an industry that is increasingly responsible for the infrastructure of everyday life.

“We really need to demystify all the algorithms that are around us,” Ullman told me as we walked down 2nd Street. “I keep hearing over and over, we want to change the world. Well: from what to what? Change it so everyone gets their dry-cleaning delivered?” Who will build the future we’ll live in? What will it look like? Right now, the important thing is that we still have a choice. “That’s why I’m advocating that everyone should learn to code at some level,” Ullman continued, “to bring in their cultural values, and their ideas of what a society needs, into this cloister.”

Many of my friends and acquaintances over the years have voiced a desire to learn to code and asked me for advice or thoughts on that path. I usually say, essentially, that “it’s not for everyone.” And then I try asking if the prospective coder has certain interests or predilections that might align with the real work of programming, the stuff you end up doing day-to-day. Did they enjoy figuring out why the VCR player or TV wasn’t working when they were growing up, for example? In asking these questions, I think that I’m trying to save the person from putting a lot of time into a skill that they might hate - like asking a person if they hate chopping onions when they’re pondering becoming a chef.

I also sometimes downplay the inherent value of learning to code. I loathe the idea in tech that code can solve all problems, or that tech will inevitably “disrupt” everything in our culture. And I worry about this narrative driving people to want to learn to code simply to stay relevant.

Ullman voices such an important and alternative view on this question. There is a narrative in the tech world that I think of as code mysticism - the idea that what programmers do is unknowable, indescribable, and unquantifiable to non-programmers. And that it can only be done by the chosen few - real hacker-y types. It’s not hard to imagine many reasons for this narrative to be perpetuated by those in power (e.g. engineers in Silicon Valley right now). And I think, as much as I hate the idea of code mysticism, when people ask me if they should learn programming, my typical response promotes that same kind of “Right Stuff” mythology.

As Anna writes, “Code, for all its elegance and power, is just a tool.” And learning the mental models and thought patterns employed by the users of that tool means you overcome that mystical protective barrier that the owners of that knowledge put up to keep out any influence from people unlike them.

For a more specific critique of the human values inherent in code right now, I would recommend Jaron Lanier’s, “You Are Not a Gadget: A Manifesto.” He argues that the current way of thinking about digital culture emerges from a philosophical foundation called cybernetic totalism, which essentially de-emphasizes people and humanity in favor of gadgets and computation.

But a simple example of the values of the programmer shaping our mental models in ways that have become assumed or implicit would be Facebook. Built by a white male programmer, Facebook basically models a person as a collection of likes. Even early on, you were asked to list your favorite movies and music and join groups organized around what you all liked.

There is nothing essential about this notion of personhood. You could model a person in many other ways - a set of experiences, dislikes, beliefs, physical traits, empathies, emotions, whatever. But this one white male programmer decided that he thought of people as the sum of what they liked, and now billions of people are heavily involved in a system that completely assumes this notion of personhood.

So, yes, if you’re at all interested in learning to code, you should do it. It’s like any other education or knowledge in that it empowers you to understand and influence more of the world. And, specifically, a world that is, as Anna writes, “increasingly responsible for the infrastructure of everyday life.” We need more perspectives and backgrounds and viewpoints influencing the models we make of the world and the technologies we produce from those models.

continue reading

History

I have begun to see the aging man’s obsession with History in myself. It is only a matter of time until my shopping cart is packed to the brim with David McCullough tomes.

I took courses on a lot of things in college: Political Science, Computer Science, Music Theory, Philosophy, Literature, Film Theory, Cultural Theory, Accounting, foreign languages, Neuroscience, Architecture, Linguistics. Some were harder than others (Music Theory was probably the toughest course I ever took), but I always felt like I understood the basic idea of the discipline - what the field was about, its foundation. It was mostly a matter of putting in the time and effort to build up the details around that foundation.

But History was the one discipline that I never quite got.

This is one of the distinguishing marks of history as an academic discipline - the better you know a particular historical period, the harder it becomes to explain why things happened one way and not another.

In fact, the people who knew the period best - those alive at the time - were the most clueless of all. For the average Roman in Constantine’s time, the future was a fog. It is an iron rule of history that what looks inevitable in hindsight was far from obvious at the time. Today is no different. Are we out of the global economic crisis, or is the worst still to come? Will China continue growing until it becomes the leading superpower? Will the United States lose its hegemony? Is the upsurge of monotheistic fundamentalism the wave of the future or a local whirlpool of little long-term significance? Are we heading towards ecological disaster or technological paradise? There are good arguments to be made for all of these outcomes, but no way of knowing for sure. In a few decades, people will look back and think that the answers to all of these questions were obvious.

(from Yuval Noah Harari’s excellent book Sapiens: A Brief History of Humankind)

I increasingly see History as the most foundational of disciplines: trying to understand an outcome as a function of Matter moving in Space and Time. As an academic discipline, it defines the “Matter” as humans and culture. We try to understand how the humans living in the 1930s moved through Space and Time in such a way that a second World War happened. But, one can think of Biology as History where the “Matter” is a plant and the outcomes are things like the conversion of light energy to chemical energy.

History is an attractive field because it takes on that most essential question we have: Why?

The above passage made me wonder about all the interest in statistics and predictive analytics today. That conundrum at the top is worth pondering - that knowledge of a sequence of events decreases your ability to make strong causal statements about it, or at least to reduce the role of randomness in an explanation. I think that’s why History is also incredibly frustrating. When looking at a sufficiently complex (and therefore interesting) system (like human culture), can you actually arrive at any satisfying answers to that essential question?

continue reading

Colophon

Note: This goes well beyond a traditional colophon into “Behind the Scenes” or “The Making Of” territory.

The reasons for redesigning one’s website are to a large degree the same as those for getting a haircut - putting one’s best (or at least a better) foot forward, aligning exterior with interior, etc. But in a redesign, there can also be value in the process itself. In that sense, it is closer to giving oneself a haircut (if that were advisable) - beneficial both in outcome and in performance.

This is an account of that process in excruciating detail. If you are not interested in design, fonts, layout, javascript, and/or web servers, and you have a tendency towards compulsively finishing things that you start, you may want to stop here.

~

The last version of my site (still available at //old.d-bow.com/) was designed in 2011 in a few hours in San Francisco with help from Adobe’s Fireworks and Ritual's wonderful coffee. I spent some time picking the color palette, designing the navigation, and incorporating a game. I did not spend time on the composition or presentation of the content. The design looked like an artless, clumsy subway poster.

I wanted my new site to focus on the content - the projects I’ve worked on, the things I’ve written, etc. And I wanted the design to get out of the way of that content. I wanted it to feel like an old book elegantly digitized.

Design

To achieve this, I focused on four particular aspects of web design: typography, color, navigation, and responsiveness.

Typography

The font was the one of the most important parts of the design since most of the content is type. I wanted a font that worked well for body copy (of which there would be quite a bit) but had a bit of quirkiness and vintage, like a weird antique. I went with Livory by HVD Fonts. I considered picking a second font for headlines, accents, etc. but decided against it in the name of simplicity (and I had no plans for elaborate navigational or decorative elements that would warrant a second font).

A moment for acknowledgements: I would never have ventured as far into the world of typography without the guidance of Meagan Fisher, who talked me through the whole process and gave me an invaluable reading list (within which the most valuable piece was this article by Jessica Hische (who, incidentally, did the illustrations for this incredible series from Penguin (a series on which, incidentally, my friend Henry worked (a circumstance that gave me the strong sense that, in choosing type, you should draw inspiration from the timelessness of classics and the warmth of friendship)))).

I should mention the drop caps and the ampersand: Meagan and Dan Cederholm wrote a blogpost many years ago on using the best ampersand, and she suggested that I use the italic Baskerville ampersand in the header. The drop caps in the “Insta-poems” section were also suggested by Meagan, and inspired by Jessica’s Daily Drop Cap project. They seemed like a nice touch given the literary, ornamental content. I spent a while sifting through the decorative fonts on Typekit before settling on Jason Walcott’s Buena Park. The “A” won me over.

Color

The palette that constitutes what I feel obligated to call “my style” tends to be fairly muted, earthy, and/or traditional, with a single surprising highlight - a small dissonance within comfortable harmony. My winter wardrobe, for example, involves a lot of unprovocative sweaters over just-visible brightly-colored t-shirts. I wanted the color palette for the site to seem very simple and comfortable: just a white background and medium-dark gray text. But to add that little highlight, the anchor tags would be a rich, saturated pink. And, finally, to bring out the headlines in the “Writing” section, there would be an almost-black gray.

The final color palette:

image

Navigation

I wanted to try something a little different with the navigation. The standard, time-honored approach is to implement a menu at the top or side of the content, with an organized collection of links that separate the different types of content. I tried a few of those designs but they felt strange in this context. That navigation makes sense for a publication or business, where the name and logo should be prominently displayed and you want to allow for clear navigation of very distinct site sections. But I wanted something a bit more personable and low key.

I ended up using a more “embedded” navigation - a sentence in the header that outlines who I am and all the things present on the site, but also serves as the way to navigate between those things. It’s essentially combining the navigation and site summary.

I like it a lot - it feels efficient and straightforward. It also makes the whole site feel more like a single piece of content than an over-engineered “interface.” However, I think it can be confusing and would probably not advocate for its use in anything but a portfolio website. It breaks from widespread convention on the web, which will most likely make it less clear to some users.

Responsiveness

For the first time, I wanted my website to totally shrink on phones. Since it’s largely text, it was very easy to constrain the content with a few media queries. The trickiest parts were updating my Tumblr layout (finding a way to include all of the extra navigational items in the minimal design) and handling the video background in the “Work” section. Video does not auto-play on phones (for good reason) and it took a little bit of trickery to ensure that mobile visitors see an alternate image instead of downloading the video. I think the last thing to do would be to blur the “Work” section images beforehand in Photoshop rather than blurring them in the browser with a CSS blur filter - it’s just extra work the browser doesn’t have to be doing (and it’s not supported in certain browsers).

Presentation

There were different challenges to presenting the content in each section of the site.

The “info” section was fairly straightforward - just a few paragraphs. But simplicity and brevity are perhaps the greatest challenges (as this post illustrates brutally).

For the “Work” section, I wanted to present the projects in a more visual way since I spend most of my time building the stuff you see in a website. I took screenshots of each project and used them as the background behind a name and a brief summary. The use of a semi-transparent background color to highlight the text in front of the images was inspired by Happy Cog’s work on visitphilly.com. A blur filter helps reduce any conflict between the text and the image (though, as mentioned above, this should be done to the underlying images). The embedding of a video from one of the projects is a dynamic highlight, though not without its costs - video makes the page a bit slower than it otherwise would be (and produces issues on mobile, outlined above).

In the “Writing” section, I wanted to present a subset of the things I’ve written over the years on my blog, but avoid conflicting “sources of truth.” I wanted the blog to remain the place for actually reading the posts, since it has useful functionality built-in (permalinks, commenting, tagging, etc.). So the “Writing” section pulls in a handful of posts and only shows roughly the first paragraph before blurring away and linking to the actual post on my blog.

Lastly, I added an “Insta-poem” section. It borders on pretense, but some of my posts on Instagram do occupy a fair bit of time and/or mental energy and I wanted a place to present them as a collection. The main challenge was deciding how much of the Instagram UI to include. I ended up creating my own simpler UI using just the image and caption, with a link to the actual post on Instagram. I wanted the site to serve as a collection of links to the actual source content, rather than a redundant location (the same reason for showing only part of the posts in the “Writing” section).

Implementation

The site was implemented with Angular and Google App Engine and the full source is here.

It uses very little of Angular’s considerable feature set - just routing, data-binding, and one API call to get the Tumblr data (posts and instapoems).

The single API call to get the Tumblr data could be moved to the server, with the response embedded in the rendered page. However, that could add a little latency to the site and I prefer to give the sense that it loads very quickly at the expense of possibly not having all the content immediately. Additionally, the content that loads dynamically is the least essential (the blogposts and instapoems, both of which can be seen on other reliable platforms).

App Engine was used to handle the game data on the old site, for which it was well-suited. In the current site, it does very little, apart from handling the Tumblr API calls and caching.

Adding a third party library to App Engine was quite tricky - probably because I’m a bit rusty with Python and there isn’t great documentation on it. After a lot of searching, I based my solution on this project, which involves a requirements.txt file to enumerate dependencies (to be installed via pip), a vendor.py file to add folders to the python path, and some code to add the ‘lib’ directory to the path in both appengine_config.py and main.py. It doesn’t feel elegant (and I’ve forgotten some of the details) but it works.

In building websites, you constantly face the choice of whether to use an existing solution or build one yourself. In fact, it’s one of the most essential parts of most software engineering. Choosing incorrectly can result in many hours spent in the most frustrating way possible. But I often feel that it’s similar to the moments in long drives where you choose between the interstate that you know might have a lot of traffic and the alternate route that might take you far out of your way (and also have traffic). You only think about the choice if you end up arriving far later than you hoped, and you immediately assume that the other option would have had a better outcome. But you never know - maybe there just wasn’t a way to drive through Washington D.C. in less than 3 hours that day.

I felt this way about the choice to use pytumblr to communicate with the Tumblr API. It seemed like the quick solution for a non-critical part of the application. Many people had spent many hours implementing a package that communicated with the API the correct way. There was no need for me to do it myself if I could just add that package and write a line of code and have what I needed. However, I spent a long time trying to figure out the third party library issue above, which made the choice seem ill-advised after the fact since I was making a single call to the API that I probably could have written manually.

The only other notable implementation detail is my use of Gulp to build the production version of the site. I spent some time learning about each module I used (in the past I’ve just blindly used the sequence of commands presented to me by a scaffold or another developer), and I commented on the commands in the gulpfile as a guide.

One particular program I used is Autoprefixer. It parses CSS and adds vendor prefixes only where necessary, based on current information from caniuse. I find that the majority of my usage of things like SASS and LESS involves mixins for vendor prefixes, nesting for organization (which I’ve come to believe creates more complexity than it eliminates, similar to how they feel at Medium), and variables for simpler maintenance of large codebases. So with Autoprefixer in place, I ended up just writing vanilla CSS.

I still think CSS preprocessors like LESS and SASS have value in larger applications, but I have been striving to avoid using a lot of the more advanced features to avoid the incidental complexity they create (hiding the true purpose of the code behind arbitrary language constructs in the name of organization or DRY-ness). Writing vanilla CSS felt simple and easy for a site like this.

~

I’m overall very pleased with how the site turned out, and even more pleased with all the things I learned along the way. Hopefully this writeup is useful to anyone thinking of doing something similar!

continue reading

Kanye Zone, Digital Signatures, and the Nature of Reality

I. They Got In Our Zone

On March 10, 2012 the “greatest video game of all time,” Kanye Zone, hit the front page of Reddit and traffic went from 1s to 100s of thousands overnight. The game has a leaderboard of top scores where people who perform superlatively at keeping Kanye out of his Zone can write a short message and post their score. As the game rapidly became more popular, this leaderboard, intended as a celebratory and civil space for high achievers, disintegrated into a cesspool of increasingly offensive messages and clearly impossible scores meant to troll the growing userbase.

Probably everyone who has made a browser-based HTML5 game has run into the problem of some users “hacking” it to post “forged” high scores. I even saw it with my personal website game. I was fortunate that my site was of little interest to anyone beyond my friends to try to attack, but with Kanye Zone we faced a much larger pool of motivated attackers who found ways around every guard we put in place. We ended up having to limit the high scores to only the most recent hour to deter people from putting in the time to figure out how to post fraudulent scores. In other words we did not prevent attacks but rather made attacks less fun and/or rewarding.

II. Cryptography and Candy Bars

[Note: Throughout this post I am thinking exclusively about JavaScript/HTML5 programs running in the browser, not compiled programs running via plugins (e.g Flash)].

It’s generally accepted that there is no way to validate with 100% certainty any output from a JavaScript program running on a malicious client. But I’ve been wondering lately if that’s actually the case, and if so, why, exactly?

The methods to generate and submit fraudulent values are fairly straightforward. At its core, any program like this must keep track of some value, and at some significant time, generate a network request to pass that value along to the server. A malicious actor can either change the value (which will then be sent along by the program), or generate its own network request that passes along its own value.

The obvious solution would be to implement some sort of digital signature whereby the server can validate that a message it receives (e.g. a new high score) was sent by a program that was run until it actually reached that state (e.g. achieved that high score). A digital signature is supposed to provide two types of assurance to a recipient of a message:

  1. That the message was actually sent by the sender (Authentication)
  2. That the message was not altered on the way (Integrity).

[Another part of #1 is that the sender cannot deny having sent it (Non-Repudiation) but that is not pertinent to this]

An analogy:

You have a friend Veronica who wants to know how many candy bars to buy at the store. She’ll buy any amount written in a letter she receives as long as it’s from you. You want her to buy 20 Hershey’s bars, but you have some chocolate-addicted friends who want her to buy way more than that. You’re worried that they will try two things to get her to buy the extra bars:

  1. Send a letter requesting 100 Hershey’s bars in an envelope with your return address.
  2. Wait at the mailbox for the letter you actually send, open it up, change the “20” to “100”, re-seal it, and put it back in the mailbox.

You need to provide Authentication to prevent their first attack, so Veronica can validate that a letter supposedly from you is actually from you, and Integrity to prevent the second attack, so she can determine that the letter she receives has not been modified in the meantime by someone else.

A simple signature in this analogy would be a scheme where every character in the letter has a different numeric value, and you simply send along the sum of all those values with your message.

As an example, say this was a part of the scheme:

character  value

     0            3

     1           10

     2            89

     3            2

To request 20 Hershey’s bars, the digital signature you’d send along would be 92 (89 + 3). You and Veronica would agree on the scheme beforehand, and only the two of you would know what each character was worth. In other words, it would be a secret you both shared. So when she received your letter with “20”, she could perform the conversion herself to make sure that it matches the 92 you sent along as the signature. Now someone else could not send a letter pretending to be you because they would not know how to generate that signature from the message. And they cannot intercept your letter and change it because then the message Veronica receives would not match the original signature you provided. Through this scheme we have achieved Authentication and Integrity and she can be confident that the letter she receives is both actually from you and has not been changed. She can buy those 20 Hershey’s bars with confidence!

III. A World Without Secrets

The problem with the conventional approach to digital signatures in the context of a client-side program is that you have to assume the entire client is compromised. A user loading your program can execute any code in any execution context, so there’s not an obvious way to keep a secret.

Digital signatures depend on some kind of shared secret. So the first thing to think about is if there is any place at all in a web page where a server can place a secret that only one specific program has access to. But as far as I can tell browsers do not currently offer this kind of security to a specific program or execution context:

  • All network traffic can be observed.
  • Closures (language-level privacy) can be accessed.
  • Same-origin policy can be bypassed.

In a world where there are no secrets, it is seemingly impossible to provide Authentication. No actor in that world has anything that is not accessible, reproducible or mutable by other actors. How would you identify yourself in a world where your social security number, appearance, even your DNA were readable and reproducible by other people? If another person existed who knew everything you knew, looked exactly like you, and had your DNA, you could not authenticate that you were the “real” you - at that moment you and the other person are identical from the point of view of any third party. In such a world, poor Veronica has no way of knowing whether one of your chocolate addict friends just took the knowledge of the numeric scheme from you and is now sending messages with your signature. In fact, they could just change your thoughts so that you actually wanted “100” Hershey’s bars.

IV. State Change-Based Messaging

What if we were to merge “Identity” and the message into a signature that is a combination of starting state and modification? Let’s say that Veronica changes her approach and decides to send you a math test, saying that for each problem solved, she’ll buy a Hershey’s bar. There would be no way for the chocolate addicts to trick her into getting more Hershey’s bars just by sending a request or intercepting the test and changing a few numbers. They could not fake it without solving the problems, and if they solve the problems then they aren’t faking it. In other words, she doesn’t care about the “realness” of the person or what they say, just that they provide something that strongly implies they have gone through the things she specifies - if they have, they are “real” or “valid.” She can’t verify that you specifically solved the problems, but she can verify that the work was done. Indeed she no longer really cares whether it’s you - her ambitions have widened and she wants to reward anyone who has solved those math problems. Identity is no longer possession of a secret, but rather evidence of having done work. The message is no longer protected by the secret, but rather contained in the work itself. The secret is the difference between the starting state (unsolved math problems) and the ending state after work (solved math problems).

Let’s return to the browser. If we cannot keep anything about our program secret to provide Authentication of its starting state (i.e. identifying the program that has our secret key), maybe we can ensure that the program end state cannot be faked (i.e. identifying the program that started this way and then did these things)? Assuming the program consists of state and functions that modify that state, we want to somehow tie the state to the sequence of function calls. For example, say our program has a variable called total that starts at 0 and a method called score that increments the total. A malicious client could just set the variable to 1000 and we would have no way of knowing that it is not equivalent to 1000 calls to score. But what if, for example, the total variable was set to a random seed from the server, and the score method hashed that seed repeatedly. There would be no way to “fake” 1000 calls to score, since there would be no way to predict what the total variable would be after 1000 calls to score except by actually performing those calls. This would provide a digital signature of sorts, where the program could send along “1000” and then the resulting hash, and the server could hash the original input 1000 times to see if it matches. If it does, it knows that the program state was actually modified 1000 times. It is a digital signature that provides Authentication and Integrity of program state changes. The message is basically “I started this way and then underwent these changes.” The secret in this instance is much more abstract. Rather than a thing that lets you generate an output that cannot be reversed to the input state (like our numeric scheme with Veronica), we have an input that cannot be converted to its output without going through the actual conversion (like solving math problems). The secret is in the conversion.

At this point we would be able to confirm that a program had started from a certain point and then gone through a sequence of specific modifications. However, a program usually has certain logical statements that determine when modifications take place. What if you wanted to confirm that those logical statements had been obeyed? As an example, say you had two squares in the program being drawn at random places on the screen and you only wanted to call the score method when those squares overlapped. How would you confirm that that program logic had been observed and the squares had overlapped 1000 times, rather than a malicious actor simply calling the score method 1000 times? Perhaps you could validate that a function that checks whether they overlap was being called X times. You could also validate the result sequence of that function by storing the result in a bitmap and sending that along with the message. And you could factor the state of the overlap function into the update to the score state each time. So if overlap is true, the score is updated one way, false a different way. With all of that in place, the server could recreate that sequence-dependent state change to validate that the result actually reflected that sequence of logical states.

V. Psychophysics

But because a client has read/write access to every execution context, we will never be able to trust that the environment or the variables and state external to the program have not been tampered with. Basically, you would have no way of knowing whether the two squares were being manipulated somehow.

In our analogy, Veronica is trying to authenticate that you know the answers to math problems to know how many Hershey’s bars to get. From your response, she can conclude that you found the answers to some number of math problems. Unfortunately, that is the only thing she can conclude. She cannot determine if you had a calculator, or if you asked another person for help, or if you actually understood the answers, or if another being momentarily inhabited your body and forced you to complete the test. We would have no way of detecting changes to the environment, inputs to the program, or things the program is supposed to measure external to it.

VI. “Woah”

When you attempt to detect artificial inputs to a program, you quickly move from security and architecture questions to more philosophical musings: What is reality? How do we know that what we see is real? What does it mean to be “human?” How do you detect “human-ness?” etc. The output of a malicious program that affects the environment / external variables (moves a mouse or moves DOM elements or mocks out parts of the environment) would have to be detected as non-human, as the work of a non-human intelligence. And we currently do not really have a good way of even understanding or conceptualizing stuff like this, let alone reliably detecting or discerning it (see Jeff Hawkins’ On Intelligence, or Brian Christian’s The Most Human Human, as two examples of great books on the topic).

So I believe this is about the best we can do to provide client-side program digital signatures. We can send out a program with a starting state, tie its state to its methods for state modification, and then confirm that a specific sequence of modifications occurred. And we can analyze the sequence to attempt to validate that it seems “legitimate” to the best of our ability, but we cannot be 100% sure that the program was being run in any specific environment. Client-side programs are a little like the people in The Matrix - they receive inputs and react to them and they can confirm that things make sense given the sequence of inputs and what they know about the world. But they have no way of knowing whether the inputs come from reality or from weird human energy harvesting pods built by machines.

VII. Some Kind Of Conclusion That Is Really Just Ending Up Where We Started But With Maybe A Newfound Understanding Of Stuff

While this feels like giving up and admitting that we can never be sure of anything (in client-side programs or in Life), this is actually a totally acceptable state of affairs. Cryptography does not ever promise 100% security and certainty. Cryptography operates in the realm of probability, a realm wholly alien to the way the brain normally perceives the world. In the original analogy for a digital signature, even if everything goes perfectly and I send Veronica the letter with my digital signature and she validates it, she can never be 100% certain that a malicious actor didn’t just guess and get it right. It’s an outcome with an incredibly small probability if your security/cryptography is designed correctly, but it is still possible.

The act of securing any system is adjusting the slider on that scale between effort and reward. You do just enough to make tampering with the system unappealing and/or unlikely to work, given the level of interest in the outcomes associated with tampering. If you were to implement a full-fledged state-modification validation scheme as I outlined for a program running in a malicious client, you would be raising the bar to tampering. A malicious actor could not easily just send fraudulent values anymore, they would have to construct a way to recreate your program’s states or interface with your program. And hopefully that’s not appealing. But if a thing is appealing enough to tamper with, malicious actors can always rise up and surpass the bar. Online poker has seen this issue with increasingly sophisticated bots created to interface with the program. Taken even further, you can imagine a scenario where the rewards of simulating a user interacting with a program are so great that a person designs a robot to sit at a desk and detect visual changes on the monitor to know how to move the mouse. The program then has the daunting task of discerning computer input from human input solely on the basis of the nature of the input. At this point, security becomes a Turing test and there is no perfect solution.

So I guess, in the end, design a sufficient level of security given the value of tampering with your program, lower the value of tampering to the best of your ability, and rest easy knowing that probably nobody would ever invest the time and money and effort to build a robot to operate your program because it’s just a silly game about Kanye West.

[Thanks to Kyle for thoughts/input on this]

continue reading

SMiLE

The myth of The Beach Boys’ SMiLE first took form in my head driving from Marin to San Francisco with my friend Matt a year or two ago. We were discussing, I don’t know, the 60s and music? I was into the Beach Boys, having re-discovered them through the eyes of the Elephant 6 collective. I was less into the Beatles but still curious. I learned of the great pop music rivalry that yielded Rubber Soul (Dec ‘65) > Pet Sounds (May '66) > Revolver (August '66) > Good Vibrations [single] (October '66), and finally Sgt. Pepper’s Lonely Hearts Club Band (June '67). How each band’s albums had been one-ups of the other’s, Pet Sounds setting the bar dominantly, only to be surpassed by Sgt. Pepper’s, which remains the pinnacle of a certain kind of upper echelon 60s pop in the collective imagination.

But there was supposed to be SMiLE, Brian Wilson’s unfinished work of probable genius that became more legendary, sublime, difficult and masterful the more its too-close-to-the-sun story of ambition and collapse took hold. I learned how Wilson had viewed SMiLE as a project that would eclipse Pet Sounds as much as Pet Sounds had eclipsed everything before it, becoming the ultimate pop masterpiece. And how hearing Sgt. Pepper’s gave him a nervous breakdown as he realized the Beatles had leapfrogged him, adding crippling and fatal stress to the SMiLE project.

We listened to a version of it I had (one of the bootleg attempts at recreating what it may have been) in the car and I remember approaching it with genuine openness. But I didn’t really appreciate it or feel like I could get much out of it at the time. I’ve been re-visiting it over the last few days and it has rewarded the time and attention. I’ve also benefited from the release of The SMiLE Sessions in 2011, which provides a fairly definitive release that I like more than the bootleg I had.

Some things I’ve read, gathered, and thought a lot about:

It was to be a “teenage symphony to God” and a "monument.“

It is about America, origin to present day, coast to coast and beyond, myth and reality.

Wilson and Parks intended Smile to be explicitly American in style and subject, a reaction to the overwhelming British dominance of popular music at the time.[4][9][nb 3] It was supposedly conceived as a musical journey across America from east to west, beginning at Plymouth Rock and ending in Hawaii, traversing some of the great themes of modern American history and culture including the impact of white settlement on native Americans, the influence of the Spanish, the Wild West and the opening up of the country by railroad and highway. Specific historical events touched upon range from Manifest destinyAmerican imperialismwestward expansion, the Great Chicago Fire, and the Industrial Revolution. Aside from focusing on American cultural heritage, Smile’s themes include scattered references to parenthood and childhoodphysical fitness, world history, poetry, and Wilson’s own life experiences. [source]

During this period, Wilson and Parks were working on an enormous canvas. They were using words and music to tell a story of America. If the early-60s Beach Boys were about California, that place where the continent ends and dreams are born, SMiLE is about how those dreams were first conceived. Moving west from Plymouth Rock, we view cornfields and farmland and the Chicago fire and jagged mountains, the Grand Cooley Damn, the California coast– and we don’t stop until we hit Hawaii. Cowboy songs, cartoon Native American chants, barroom rags, jazzy interludes, rock'n'roll, sweeping classical touches, street-corner doo-wop, and town square barbershop quartet are swirled together into an ever-shifting technicolor dream. [source]

It is a profoundly psychedelic album, but with a darker focus. Where the Beatles seemed to view psychedelics as a liberating path to insight and love, or something, SMiLE fixates on the mental break inherent in the experience:

Amid the outtakes – stuff even the most dedicated bootleg collector won’t have heard – there’s a telling moment. "You guys feeling any acid yet?” asks Wilson during a session for Our Prayer. He sounds terrified. LSD didn’t make Wilson relax and float downstream: it scared the shit out of him, lighting the touch paper on a whole range of mental health problems. And he couldn’t keep the fear out of his music. Even at its most remorselessly upbeat, the Beach Boys’ music was marked by an ineffable sadness – you can hear it in the cascading tune played by the woodwind during Good Vibrations’s verses – but on Smile, the sadness turned into something far weirder. All the talk of Wilson writing teenage symphonies to God – and indeed the sheer sumptuousness of the end results – tends to obscure what a thoroughly eerie album Smile is. Until LSD’s psychological wreckage began washing up in rock via Skip Spence’s Oar and Syd Barrett’s The Madcap Laughs, artists tactfully ignored the dark side of the psychedelic experience. But it’s there on Smile: not just in the alternately frantic and grinding mayhem of the instrumental Fire, but in Wind Chimes’s isolated, small-hours creepiness and the astounding assembly of weird, dislocated voices on I Love to Say Da Da.

Isolated, dislocated, scared: Smile often sounds like the work of a very lonely man. There’s not much in the way of company when you’re way ahead of everyone else. Whether anyone would have chosen to join Wilson out there in 1967 seems questionable: perhaps they’d have stuck with the less complicated pleasures of All You Need Is Love after all. What’s beyond doubt is the quality of the music he made. [source]

My favorite line from that review is “It was a psychedelic experience Hendrix wouldn’t understand, but Don Draper might.” I tend to agree with that review’s conclusion that even if it had been released in 1967, it would probably not have Changed Music History so much as been the book-end to a particular pop music trajectory, the ultimate square psychedelic pop album that maybe went a bit too far. Perhaps a “normal” American mind warped by psychedelics is just more disturbing than a mind that already lives on the outskirts?

Wilson had gone a tad off the deep end. Mary Jane had long been a friend, but Wilson had grown increasingly close with Lucy (we mean acid, folks). This both allowed for the extreme insights he had during recording and began to dissemble the already overworked and paranoid Wilson. He became convinced the “Fire” section of ‘Elements Suite’ was responsible for local fires, ceasing recording of the material, had his band dress in firemen hats, and so on. [source]

There are some profoundly beautiful moments, and some profoundly eerie and disturbing moments. “Our Prayer” is beautiful. “Do you like worms” is deeply eerie, especially the harpsichord part. I really love “Wind Chimes” and “Surf’s Up.” “Heroes and Villains” had/has “Good Vibrations”-style musical ambitions and was supposed to be the thematic centerpiece, but it falls short for me in a way I can’t quite put my finger on.

The story/legend/myth of SMiLE certainly adds a lot to the experience of it. And the musical/lyrical experience alone is every bit as demanding and layered and ambitious as legend had it. But I think we wouldn’t have been quite as willing to live in and explore that warped pop drug vision of America as we maybe assumed.

continue reading

JS Drama Reading List or: How I Learned to Stop Worrying and Love the Semicolon

A couple months ago, there was a lot of drama in the javascript world.  I thought the anger and intensity of the debate was really silly, but I think it did do a good job of prompting a lot of frontend engineers (me included) to revisit and brush up on automatic semicolon insertion (which was a good thing).  I finally got around to reading all the things on it in my todo list, and wanted to share some of the best:

My ultimate opinion was something along the lines of what Brendan Eich said:

“The moral of this story: ASI is (formally speaking) a syntactic error correction procedure. If you start to code as if it were a universal significant-newline rule, you will get into trouble.”

I agree that it’s important to understand this part of the language.  But I personally would not think of it as a “feature” but rather as a “safeguard,” and treat it as such.  Also, I always err on the side of obviousness and simplicity, rather than minimalism and trickiness.  If I want to end a statement, I want to explicitly end it with the semicolon.  This feels less error-prone for me as the writer, AND easier for people reading my code to know my intention.  I think the readability aspect was downplayed by the people arguing for not using semicolons.  I find the Bootstrap code tougher to read because I’m not used to semicolon-less code.  This issue is sort of hilariously brought up here.  But in that group discussion, I think they still ignore the readability issues and the benefits of having a common style in the community (the discussion really only points to a sort of territoriality as justification for adopting a given style).

The whole argument sort of felt like ASI is a minefield, and there are some people who know how to avoid the mines and can go straight across with minimal risk; but there are others who don’t know how to spot the mines that well, and instead opt for a slightly longer route that everyone tells them to take because it’s safer.  I don’t think that’s necessarily an irresponsible decision (not to learn to spot the mines when there is a route that bypasses them).  But, again, the best thing to come out of this was probably alerting a lot of developers that there is a way to spot those mines and it’s probably worth learning it, independent of which style you want to use.

continue reading

Paying for music: my 2 cents (pun intended)

Credibility-establishing Editorial Note: I have paid for all my music in at least the last 5 years.  I relish my trips to my favorite local record store to (gasp) buy CDs, and I use several music services that are “legitimate” (like Spotify, Turntable, Pandora, and iTunes).

Like everyone else on the Internet lately, I read this blog post arguing in favor of people paying for music recordings to support artists.  But rather than argue against any of the specific points (a temptation very difficult to resist, especially when he seemingly tries to argue that my CD burning activity in high school was to blame for several suicides), I want to think more about the fundamental question: “Why don’t people want to pay for recorded music?”

In a lot of these anti-piracy arguments, there’s the assumption that artists are entitled to monetary compensation for their recordings.  I believe this assumption is now false, and that’s the real reason for the changes in how people acquire and consume recorded music.

The assumption that artists should be compensated with money for recording music is probably based on economic and technological factors in the 20th century, when the means to produce and distribute quality music were limited to “professionals.”  Recording equipment required experience and expertise to produce anything listenable.  Additionally, getting recordings into the hands of a large audience required a lot of infrastructure - creating the physical albums, shipping them, marketing them, etc..  When the production and distribution of music had such barriers to entry, recorded music had economic value and people were willing to pay for the music that made it through those barriers.  Musicians were operating in an industry, producing something that only a limited set of people had the means to produce.

So what is “economic value?”  I’m using it to mean the perceived monetary exchange rate for a given good or service.  And it is usually predicated on that good or service not being widely accessible - basic supply and demand.  This has always been the case.  If I were a cabinetmaker, I would never pay for someone else to make me a cabinet because it’s a service directly accessible to me - I would have the requisite expertise, means, and desire to perform the task such that the economic value of someone else doing it would be well below the amount I would pay.  But other people are farmers or lawyers or plumbers, and they do not have the expertise or means to make cabinets.  So they would pay me to produce cabinets.  Because the supply of cabinets is low, the supply of cabinet-making skills is low, and the demand for cabinets is high, those cabinets and skills have more economic value.

Have you used music production software lately?  It’s really easy to make a professional sounding recording.  And once you do, it’s super easy to post the file online and then anyone in the world with an internet connection can hear it.  In my mind, music production no longer has the barriers to entry that give it economic value.  Amateurs are often making music that I like more than “professional” recordings.  Heck, even I can make music that I like listening to more than a lot of “professional” recordings.  In my view, recording music no longer carries economic value, so I generally don’t feel the need to compensate musicians with money for their recordings.

If this seems cruel or arrogant or ridiculous, consider this passage from This is Your Brain on Music: The Science of a Human Obsession (bolding added by me):

Music is unusual among all human activities for both its ubiquity and its antiquity.  No known human culture now or anytime in the recorded past lacked music.  Some of the oldest physical artifacts found in human and protohuman excavation sites are musical instruments: bone flutes and animal skins stretched over tree stumps to make drums. […]  Even more so in nonindustrialized cultures than in modern Western societies, music is and was part of the fabric of everyday life.  Only relatively recently in our own culture, five hundred years or so ago, did a distinction arise that cut society in two, forming separate classes of music performers and music listeners.  Throughout most of the world and for most of human history, music making was as natural an activity as breathing and walking, and everyone participated.  Concert halls, dedicated to the performance of music, arose only in the last several centuries.

Jim Ferguson, whom I have known since high school, is now a professor of anthropology.  […]  For his doctoral degree at Harvard, he performed fieldwork in Lesotho, a small nation completely surrounded by South Africa.  There, studying and interacting with local villagers, Jim patiently earned their trust until one day he was asked to join in one of their songs.  So, typically, when asked to sing with these Sotho villagers, Jim said in a soft voice, “I don’t sing,” and it was true: We had been in high school band together and although he was an excellent oboe player, he couldn’t carry a tune in a bucket.  The villagers found his objection puzzling and inexplicable.  The Sotho consider singing an ordinary, everyday activity performed by everyone, young and old, men and women, not an activity reserved for a special few.

Our culture, and indeed our very language, makes a distinction between a class of expert performers - the Arthur Rubinsteins, Ella Fitzgeralds, Paul McCartneys - and the rest of us.  The rest of us pay money to hear the experts entertain us.  Jim knew that he wasn’t much of a singer or dancer, and to him, a public display of singing and dancing implied he thought himself an expert.  The villagers just stared at Jim and said, “What do you mean you don’t sing?! You talk!”  Jim told me later, “It was as odd to them as if I told them that I couldn’t walk or dance, even though I have both my legs.”  Singing and dancing were a natural activity in everybody’s lives, seamlessly integrated and involving everyone […]

The idea of recorded music having economic (monetary) value arose after the means to produce it (at the quality level our culture came to expect) became scarce.  And when something acquires monetary value, an Industry builds up around it.  But that economic value is eroding as the means to produce it are made ubiquitous.

I think this idea explains the differences in how other content industries are faring right now.  The means to produce and distribute print content have also been made more accessible, and you see the newspaper and magazine industries struggling in the same way as the music industry.  (Obviously there are nuances to that whole discussion that I won’t go into).

But Hollywood and the TV industry have not been affected as much because very few “amateurs” have the means to produce the same quality of content.  I’m willing to pay for “Prometheus” because the industry adds value to that movie - can you even imagine how awful an “amateur” version of that movie would be?  But, if software is invented that allows for really simple and compelling effects / CGI, audio editing, camera work, lighting, etc. I could imagine not feeling like Hollywood was adding any economic value anymore and not wanting to pay for movies.

I think another important point here is that these content industries have always involved an uneasy fusion of Art and Capitalism, and that given the choice, people want to lean towards Art.  Put another way, people prefer to think of music as an expression of something emotional and visceral that just wants to get out into the world, not as an industry or a business or a vehicle to make money.  When anti-piracy advocates talk about not paying musicians as a “social injustice,” it simply doesn’t hit home.  You have to pick a side - is music to be treated as Art?  If so, then expecting payment is just delusional.  Is music then a business?  If so, the more broadly accessible means of production and distribution have eliminated the economic value that made recording viable as an industry.  Deal with it.  We didn’t like music being a business anyway.

This paragraph in the blog post highlights how little anti-piracy advocates seem to be grasping the real issue:

The fundamental shift in principals and morality is about who gets to control and exploit the work of an artist. The accepted norm for hudreds of years of western civilization is the artist exclusively has the right to exploit and control his/her work for a period of time. (Since the works that are are almost invariably the subject of these discussions are popular culture of one type or another, the duration of the copyright term is pretty much irrelevant for an ethical discussion.) By allowing the artist to treat his/her work as actual property, the artist can decide how to monetize his or her work. This system has worked very well for fans and artists. Now we are being asked to undo this not because we think this is a bad or unfair way to compensate artists but simply because it is technologically possible for corporations or individuals to exploit artists work without their permission on a massive scale and globally. We are being asked to continue to let these companies violate the law without being punished or prosecuted. We are being asked to change our morality and principals to match what I think are immoral and unethical business models.

There is no immorality or ethics here!  The changes in business model are not caused by morals or ethics, but rather by economics.  When you take Art and monetize its production and create an industry, and that industry is then faced with the realities of capitalism and supply and demand, don’t act like there is social injustice at work.  You can object to the capitalist system, but you can’t argue that things should work the way they did when capitalism was going well for you.

I think the bottom line is that nobody cares if the Music Industry dies because nobody believes it is adding value anymore, and that Music will continue on just as it has since the protohumans first started making it.  That’s why people won’t pay for it.

Follow-up to Editorial Note: So if I don’t believe artists are entitled to be paid, why do I buy music still?  Well, when I buy a CD, the value I perceive is not necessarily that the artist and the industry created this music I’m about to hear.  The value is in the experience of going to the record store; the value is the record store itself, which curates and highlights music I wouldn’t normally find (“discovery” value); the value is in supporting the employees who will filter out the noise.  And I use services like Spotify or iTunes not because I believe I’m compensating artists, but because they give me access to music instantly and easily (streaming!!!).  I use Turntable because it’s a unique social experience.  In short, I don’t pirate music, but not because I believe artists are entitled to money.  Rather I place economic value on other things around music.  And, lastly, I don’t object to artists making money.  But I don’t think any artist should feel like they are entitled to be paid for their recorded music - I don’t think it should be viewed as an economic good or product.

EDIT: A great response to this.  And my response to that response.

continue reading

Not all data is created (or maintained) equally

I had a great meeting this morning with two people doing amazing work in San Francisco documenting and disseminating information on homeless resources.  We’re hoping to partner with them for www.projectOPENsf.org to digitize their resource and present the data in new and useful ways.  We’ve been discussing this project with a lot of different folks all over the city for a long while now and this meeting got me reflecting on some things I’ve learned from the project about different types of data sets.

Currently we’re working with a great wiki site that many people highlighted as one of the more reliable and up-to-date resource listings.  The vision for this wiki (and for any wiki I suppose) is to have the community create and maintain the data set.  This has resulted in a very comprehensive resource, with over 1000 information-rich pages.  And theoretically, each resource provider will maintain their own information, so the resource as a whole doesn’t depend on any individual organization or person.  However community-maintained resources (inevitably?) have reliability issues - If nobody is explicitly in charge, it’s hard to incentivize keeping things up-to-date.  And there isn’t an over-arching structure enforced, so it can be hard to parse the information and extract insights.

The people we talked to today are on the other end of the spectrum.  They do a ton of legwork each quarter to personally reach out to every resource they list and validate their information.  Their publication is much smaller in terms of quantity (~200) but is probably among the most high quality data sets in existence for San Francisco homeless resources.  It has a small, dedicated group of people who put in a lot of work to structure and update the information.  But without that group, the resource would most likely cease to exist or be valuable.

A major theme of my non-profit volunteer work has been bringing technology (computation, data, and design) to spaces outside of where we (the volunteers) usually see it (our Silicon Valley jobs).  At our day jobs, the data sets are usually highly structured, and easily and automatically maintained - revenue figures and pageviews and application interactions and logs, etc.  With data sets like that, the challenge is usually around tying them together in interesting ways and cutting through the vast quantity of information to extract the actionable macro insights.  But the data sets I encounter outside my day job are often unorganized, unstructured, and/or unmaintained.  With these data sets, there’s a foundational problem to solve before you can do anything else - how do you organize the data and how can you maintain that structure in the future?  Once you’ve solved that, the actual parsing and presenting of that information is almost an afterthought - the easy part!

It seems the best solution lies in some sort of combination of community-maintenance and individual-maintenance.  I’m increasingly convinced that there’s a certain human element to it - a passionate, interested individual or organization that will articulate and enforce a structure/vision for the data set.  Silicon Valley tends to believe that technology can solve every problem, but there are definitely some problems that are better suited to a technical solution than others.  Computers solve pure data problems, fundamentally.  Other problems, like how you ensure that a person experiencing homelessness is able to find the resource they need, have so many components outside of the simple data problem (inputting a “need” and parsing a data set to return the result) that you can never entirely remove the passion and dedication of an individual from the solution.  Maybe technology’s role in these problems is to enable those people to channel their energy and passion efficiently and effectively, to connect with one another, and to parse and present the results of their labor.

Overall, after talking to people in different service organizations and government entities, the one thing that’s clear is that nobody has the perfect solution yet.  The best thing we can do is to keep thinking and brainstorming and trying different things, which is what we’re hoping to do with www.projectopensf.org!

continue reading