PDF Metadata

(The exact PDF I used has the SHA-256 hash: 3d0d66be9587018082b41f8a676c90041fa2ee0455571551d266e4ef8613b08a².)

The second thing I did was take a look at the metadata³:

• The creator tool checks out: “DynamicPDF v5.0.2 for .NET” is part of a commercial suite, and it was pirated well before April 2009_15ya, although I could not figure out when the commercial release was.

• The date, though, is “Thu 2009-04-09 09:32:47 PM EDT”. Keep in mind, this leak was in May-October 2009_15ya, and the original Variety announcement was dated 2009-04-30.

  If one were faking such a script, wouldn’t one through either sheer carelessness & omission or by natural assumption (the Parlapanides signed a contract, the press release went out, and they started work) set the date well after the announcement? Why would you set it close to a month before? Wouldn’t you take pains to show everything is exactly as an outsider would expect it to be? As Jorge Luis Borges writes in “The Argentine Writer and Tradition”:

  Gibbon observes [in the Decline and Fall of the Roman Empire] that in the Arab book par excellence, the Koran, there are no camels; I believe that if there were ever any doubt as to the authenticity of the Koran, this lack of camels would suffice to prove it Arab. It was written by Mohammed, and Mohammed as an Arab had no reason to know that camels were particularly Arab; they were for him a part of reality, and he had no reason to single them out, while the first thing a forger or tourist or Arab nationalist would do is to bring on the camels - whole caravans of camels on every page; but Mohammed, as an Arab, was unconcerned. He knew he could be Arab without camels.

  Another small point is that the date is in the “EDT” timezone, or Eastern Daylight-savings Time: the Parlapanides have long been based out of New Jersey, which is indeed in EDT. Would a counterfeiter have looked this up and set the timezone exactly right?

Writing/formatting

What of the actual play? Well, it is written like a screenplay, properly formatted, and the scene descriptions are brief but occasionally detailed like the other screenplays I’ve read (such as the Star Wars trilogy’s scripts). It is quite long and detailed. I could easily see a 2 hour movie being filmed from it. There are no red flags: the spelling is uniformly correct, the grammar without issue, there are few or no common amateur errors like confusing “it’s”/“its”, and in general I see nothing in it - speaking as someone who has been paid on occasion to write - which would suggest to me that the author(s) were neither of professional caliber nor unusually skilled amateurs.

The time commitment for a faker is substantial: the script is ~22,000 words, well-edited and formatted, and reasonably polished. For comparison, NaNoWriMo tasks writers with producing 50,000 words of pre-planned, unedited, low-quality content in one month, with a second month (NaNoEdMo) devoted to editing. So the script represents at a minimum a month’s work - and then there’s the editing, reviewing, and formatting (and most amateur writers are not familiar with screenwriting conventions in the first place).

So much for the low-hanging fruit of internal evidence: all suggestive, none damning. A faker could have randomly changed Charles to “Charley”, looked up an appropriate address, edited the metadata, come up with all the Hollywood touches, wrote the whole damn thing (quite an endeavour since relatively little material is borrowed from DN), and put it online.

Stylometrics

The next step in assessing internal evidence is hardcore: we start running stylometry tools on the leaked script to see whether the style is consistent with the Parlapanides as authors. The PDF is 112 images with no text provided; I do not care to transcribe it by hand. So I split the PDF with pdftk to upload both halves to Google Docs (which has an upload size limit of 1.02m characters) to download its OCR’ed text; and then ran the PDF through GOCR to compare - the Google Docs transcript was clearly superior even before I spellchecked it. (In a nasty surprise halfway through the process, I found that for some reason, Google Docs would only OCR the first 10 pages or so of an upload - so I wound up actually uploading 12 split PDFs and recombining them!)

Samples of the Parlapanides’ writing is hard to obtain; the only produced movie with their script is the 2000_24ya Everything For A Reason and the 2011_13ya Immortals (so any analysis in 2009_15ya would’ve been difficult). I could not find the script for either available anywhere for download, so I settled for OpenSubtitles.org’s subtitles in .srt format and stripped the timings: grep -v [0-9] Immortals.2011.DVDscr.Xvid-SceneLovers.srt > 2011-parlapanides-immortals.txt (There are no subtitles available for the other movie, it seems.)

Samples of fanfiction are easy to acquire. FanFiction.Net’s Death Note section (24,246 fanfics), sort by: number of favoriting users, completed, in English, and >5000 words. This yields 2,028 results but offers no way to filter by fanfictions written in a screenplay or script style, and no entry in the first 5 pages mentions “script” or “screenplay” so it is a dead end. The dedicated play/musical section lists nothing for “Death Note”. Googling "Death Note" (script OR screenplay OR teleplay) -skit site:fanfiction.net/s/ offers 8,990 hits, unfortunately, the overwhelming majority are either irrelevant (eg. using “script” in the sense of cursive writing) or too short or too low quality to make a plausible comparison. (I also submitted a Reddit request, which yielded no suggestions.) The final selection:

As a control-control, I selected some fanfictions that I knew to be of higher quality:

The fanfictions were converted to text using the now-defunct Web version of FanFictionDownloader.

With 10 fanfictions, it makes sense to compare with 10 real movie scripts; if we didn’t include real movie scripts formatted like movie scripts, one would wonder if all the stylometrics was doing was putting one script together with another. So in total, this worry is diluted by 3 factors (in descending order):

1. the use of 10 real movie scripts (as just discussed)

2. the use of 10 fanfictions resembling movie scripts to various degrees (previous)

3. the known Parlapanides work (the Immortals subtitles) being pure dialogue and including no action or scene description which the stylometrics could “pick up on”

The scripts, drawn from a collection (grabbing one I knew of, and then selecting the remaining 9 from the first movies alphabetically to have working .txt links as a quasi-random sample):

For the actual analysis, we use the computational stylistics package of R code; after downloading stylo, the analysis is pretty easy:

install.packages("tcltk2")
source("stylo_0-4-6_utf.r")

The settings⁴ are to: run a cluster analysis which uses the entire corpus, assumes English, and looks at the difference between files in their use of “most popular words” (starting at 1 word & maxing out at 1000 different words, because the entire Immortals subs is only ~4000 words of dialogue), where difference is a simple Euclidean distance.

The script PDF, full corpus, intermediate files, and stylo source code are available as a tarball.

The graphed results are unsurprising:

1. The movies cluster together in the top third

2. The DN fanfics are also a very distinct cluster at the bottom

3. In the middle, splitting the difference (which actually makes sense if they are indeed more competently or “professionally” written), are the “good” fanfics I selected. In particular, the fanfics by Eliezer Yudkowsky are generally close together - vindicating the basic idea of inferring authorship through similar word choice.

4. Exactly as expected, the Immortals subs and the leaked DN script are as closely joined as possible, and they practically form their own little cluster within the movie scripts.

   This is important because it’s evidence for 2 different questions: whether the known Parlapanides work is similar to the leaked script, and whether the leaked script is similar to any fanfictions rather than movies. We can answer the latter question by noting that it is grouped far away from any fanfiction (the only fanfiction in the cluster, the “Three Characters” fanfiction, is very short and formalized), even though Eliezer Yudkowsky (himself a published author) wrote several of the fanfictions and one of them (Harry Potter and the Methods of Rationality) is intended for publication and perhaps even a Hugo award.

That the analysis spat out the files together is evidence: there were 30 files in the corpus, so if we generated 15 pairs of files at random, there’s just a 1⁄15 = 6.6% chance of those two winding up together. The tree does not generate purely pairs of files, so the actual chance is much lower than 6.6% and so the evidence is stronger than it looks; but we’ll stick with it in the spirit of conservatism and weakening our arguments.

Dating

But is there any external evidence? Well, the timeline is right: hired around June 2008_16ya, delivered a script in early April 2009_15ya, official announcement in late April 2009_15ya. How long should delivery take? The interval seems plausible: Figure about 2 months for both brothers to read through the DN manga or watch the anime twice, clear up their other commitments, a month to brainstorm, 3 months to write the first draft, a month to edit it up and run it by the studio, and we’re at 7 months or around February 2009_15ya. That leave a good 6 months for it to float around offices and get leaked, and then come to the wider attention of the Internet.

Credit

Given this effort and the mild news coverage of it, one might expect a faker to take considerable pride in his work and want to claim credit at some point for a successful hoax. But as of January 2013_11ya, I am unaware of anyone even alluding or hinting that they did it.

Official Statements

Additional evidence comes from the January 2011_13ya announcement by Warner Bros that the new director was one Shane Black, and the script was now being written by Anthony Bagarozzi and Charles Mondry (with presumably the previous script tossed):

“It’s my favorite manga, I was just struck by its unique and brilliant sensibility,” Black said. “What we want to do is take it back to that manga, and make it closer to what is so complex and truthful about the spirituality of the story, versus taking the concept and trying to copy it as an American thriller. Jeff Robinov and Greg Silverman liked that.” Black’s repped by WME and GreenLit Creative.

ANN quoted Black at a convention panel:

However, Black added that the project was in jeopardy because the studio initially wanted to lose “the demon [Ryuk]. [They] don’t want the kid to be evil… They just kept qualifying it until it ceased to exist.” Black said that “the creation of a villain, the downward spiral” of the main character Light has been restored in the script, and added that this is what the film should be about.’

…According to the director of Kiss Kiss Bang Bang and the upcoming Iron Man 3 film, the studios initially wanted to give the main character Light Yagami a new background story to explain his “downward spiral” as a villain. The new background would have had a friend of Light murdered when he was young. When Light obtains the Death Note - a notebook with which he can put people to death by writing their names - he uses it to seek vengeance. However, Black emphasized that he opposed this background change and the suggested removal of the Shinigami (Gods of Death), and added that neither change is in his planned version.

Black’s comments line up well with the leaked script: Ryuk is indeed omitted entirely, Light is indeed mostly good and redeemed, Light does have a backstory justifying his vengeance, and so on. The only discordant detail is that in the leaked script, it was his mother murdered and not “a friend”.

Legal Takedowns

The original Anime Vice article had commenters provide two sendspace links for downloading the script. Both files went dead quickly, and an uploader wrote “WB took it down that proves it is not fake”. The sendspace links merely say that the files are no longer available, without giving any explicit reason like a DMCA takedown.

Assuming it was a DMCA takedown, who did it? Not the 2 brothers, who might have a legal right to order the takedown of material falsely attributed to them (I am not clear on the remedies available for a false attribution of authorship), but surely either the commissioning studio or their partner. Needless to say, they do not have a standing RIAA-style war against DN fanfiction or fan-art or even torrents of the anime or scanlations of the manga; just this script. (Possibly if the script were not the studio’s property, it wouldn’t have any legal ground to demand takedowns - their license likely covers just the movie rights, and so fanfiction in the form of a script (for example) would infringe on the Japanese rights-holder, not the studio.)

The same uploader says:

I called Warner Bros After all the channels at WB, i finally got to the WB backed company Producing DN Dan Lin Pictures i gave them a chance to clear the air on the leaked script, to prove or disprove it they said “no comment at this time”

The original Anime Vice author wrote

Rather than run with the story then, I called Lin Pictures to see if they could confirm or deny whether the script was actually theirs or a fan-written phony. I was told I would get a call back, but never did. I tried calling back a second time earlier this week, this time passing on considerably more information, and still no call back. As such, I have come to the conclusion that the company isn’t overly concerned with the script, which suggests several possibilities to me:

1. It’s not a legitimate script at all, so they’re not worried about it.

2. It’s an old draft…different from the current version, so they’re not worried about it.

3. The script was intentionally leaked for promotional purposes or to gauge fan reaction.

I don’t buy it. If it is a fake script, why not simply deny it - either time? It is not as if companies usually have any trouble denying things. A “no comment” is more consistent with it being real and them also sending takedowns.

I find this external legal evidence compelling, and in conjunction with the internal evidence and oddities best explained by the leaked script being really by said Hollywood scriptwriters, I believe the script real. Perhaps an early draft to be discarded or rewritten, but still genuine. I suppose an American DN movie could be much worse: just consider Dragon Ball Evolution or The Last Airbender!

Authorship

How would we estimate the evidence of “Charley Parlapanides”? The names of the writers could either be:

1. present and wrong

   Very strong evidence it is fake: who puts their own name down wrong? This would be overwhelming evidence, but we don’t have it so we will drop this possibility from consideration and consider the remaining possibilities:

2. present and right

   Evidence it is real. Of the 10 scripts used in the stylometric, 9⁄10 included right authorship information.

3. not present

   Of the 4 known fake scripts mentioned previously, only 2 included authorship information.

Given this information, how does the presence of right authorship influence our prior belief of 50%?

Let a be “is real” and b be “has correct authorship”. We want to know the probability of a given the observation “correct authorship”. A version of Bayes’s theorem (stolen from “An Intuitive Explanation of Bayes’s Theorem”; you can see other applications in my modafinil essay; a nice visualization is given by Oscar Bonilla or one could watch distributions be updated):

$$P\left(a|b\right)=\frac{P\left(b|a\right)\cdot P\left(a\right)}{\left(P\right(b|a)\cdot P(a\left)\right)+\left(P\right(b|\neg a)\cdot P(\neg a\left)\right)}$$

If you look, the right-hand side of that equation has exactly 4 pieces in its puzzle:

1. $P\left(a\right)$

   This is something we already know, “probability of being real”. This is the base rate we already estimated at 50% or 0.5.

2. $P\left(\neg a\right)$

   This is the negation of the previous. What is the negation of 50%, its contrary? 50%.

3. $P\left(b|a\right)$

   Remember, we read the pipe notation backwards, so this is ‘the probability that a real script (a) will include authorship’ (b)’. We said that 9⁄10 of good scripts include authorship, so this is 90% or 0.9. (One way to compensate for the small sample size of 10 scripts would be to use Laplace’s rule of succession, $\frac{n+1}{m+2}$, which would yield $\frac{9+1}{10+2}=0.83$.)

4. $P\left(b|\neg a\right)$

   Finally, we have “the probability that a fake script will include authorship”. We looked at 4 fake scripts and 2 included authorship, which is another 50% or 0.5.

To put all these definitions in a list:

1. a = is real

2. b = has authorship

3. $P\left(a\right)$ = probability of being real = 50% = 0.50

4. $P\left(\neg a\right)$ = probability of being not real = 50% = 0.50

5. $P\left(b|a\right)$ = probability a real script will include authorship = 90% = 0.9

6. $P\left(b|\neg a\right)$ = probability a fake script will include authorship = 50% = 0.5

We substitute in to the original equation:

$$P\left(a|b\right)=\frac{P\left(b|a\right)\cdot P\left(a\right)}{\left(P\right(b|a)\cdot P(a\left)\right)+\left(P\right(b|\neg a)\cdot P(\neg a\left)\right)}=\frac{0.9\cdot 0.5}{(0.9\cdot 0.5)+(0.5\cdot 0.5)}=\frac{0.45}{0.45+0.25}=\frac{0.45}{0.7}=0.643$$

Sanity checks:

1. Authorship is evidence for it being real; did we increase our confidence that the script is real?

   Yes, because 64.3% > 50%. So we moved the right direction.

2. Did we move the right amount?

   Well, the fake scripts have a 50% rate and the real scripts have 90%; since this is the only evidence we’ve taken into account so far, our first calculation shouldn’t move us “very far”, whatever that means, since not all real scripts have authorship and plenty of fake ones are careful to include them. (Imagine a world where 80% of fakes include authorship: authorship would become even weaker evidence; and when fakes hit 90% inclusion, authorship would be so weak as to be no evidence at all since the fakes and reals look exactly the same.) The inclusion of authorship does not seem like tremendous evidence so after taking authorship into account, we should be close to our original prior of 50% than to any extreme certainty like 90%.

   Are we? Our posterior of 64% doesn’t strike me as a big shift from 50%, so we conclude that this second sanity check is satisfied. Good!

A final calculation: the probability that “a test gives a true positive” divided by “the probability that a test gives a false positive” ($\frac{P\left(b|a\right)}{P\left(b|\neg a\right)}$) is the “likelihood ratio” of that test (see also odds ratio). A likelihood ratio of 1 indicates that our test is useless as it is equally likely for real scripts and fake scripts alike; <1 indicates it is evidence against being real, and >1 evidence for being real. Likelihood ratios will be useful later, so we’ll calculate them too as we go along. So:

$$\frac{P\left(b|a\right)}{P\left(b|\neg a\right)}=\frac{0.9}{0.5}=1.8$$

(As expected of evidence for the script being real, the likelihood ratio > 1.)

Dating

The argument there seems to be of the form that a PDF dated April 2009_15ya is consistent with the estimated timeline for the true script. But what would be inconsistent? Well, a PDF dated after April 2009_15ya: such a PDF would raise the question “what exactly the brothers were doing from June 2008_16ya all the way to this counterfactual post-April 2009_15ya date?”

But it turns out we already used this argument! We used it as the PDF date inversion test. Can we use the April date as evidence again and double-count it? I don’t think we should since it’s just another way of saying “April and earlier is evidence for it being real, post-April is evidence against”, regardless of whether we justify pre-April dates as being during the writing period or as being something a faker wouldn’t dare do. This argument turns out to be redundant with the previous internal evidence (which in hindsight, starts to sound like we ought to have classified it as external evidence).

What we might be justified in doing is going back to the PDF date inversion test and strengthening it since now we have 2 reasons to expect pre-April dates. But as usual, we will be conservative and leave out this strengthening.

Credit

This is an interesting external argument as it’s the only one dependent purely on the passage of time. It’s a sort of argument from silence, or more specifically, a hope function.

Credit & Hope Function

In the case of “taking credit”, we can imagine the boxes as years, and each year passed is a box opened. As of October 2012_12ya, we have opened 3 boxes since the May/October 2009_15ya leak. How many boxes total should there be? I think 20 boxes is more than generous: after 2 decades, the DN franchise highly likely won’t even be active⁸ - if anyone was going to claim credit, they likely would’ve done so by then. What’s our prior probability that they will do so at all? Well, of the 4 faked scripts, the author of the Mr. Peepers script took credit but the other 3 seem to be unknown - but it’s early days yet, so we’ll punt with a 50%. And of course, if the script is real, very few people are going to falsely claim authorship (thereby claiming it’s fake?). So our setup looks like this:

1. a = is real

2. b = no one has claimed authorship

3. $P\left(a\right)$ = probability of being real = 0.899

4. $P\left(\neg a\right)$ = probability of being not real = 1 - 0.899 = 0.101

5. $P\left(b|a\right)$ = probability a real script will have no ownership claim = 99% (shit happens⁹) = 0.99

6. $P\left(b|\neg a\right)$ = probability a fake script will have no ownership claim = probability someone will claim it is the hope function with n = 20, i = 3, L₀ = 50% = $\frac{\frac{20-3}{20}\cdot 0.5}{\frac{20-3}{20}\cdot 0.5+(1-0.5)}=0.45945$, so the probability someone will not is $1-0.45945=0.54055$

Then Bayes:

$$P\left(a|b\right)=\frac{P\left(b|a\right)\cdot P\left(a\right)}{\left(P\right(b|a)\cdot P(a\left)\right)+\left(P\right(b|\neg a)\cdot P(\neg a\left)\right)}=\frac{0.99\cdot 0.899}{(0.99\cdot 0.899)+(0.54055\cdot 0.101)}=0.942$$

Likelihood ratio:

$$\frac{0.99}{0.54055}=1.831$$

Official Statements

The 2011 descriptions of the plot of the real script match the leaked script in several ways:

1. no Ryuk or shinigamis

   This is an interesting change. I don’t think it’s likely a faker would remove them: without them, there’s no explanation of how a Death Note can exist, there’s no comic relief, some plot mechanics change (like dealing with the hidden cameras), etc. Certainly there’s no reason to remove them because they’re hard to film - that’s what CGI is for, and who in the world does SFX or CGI better than Hollywood?

2. Light ends the story good and not evil

3. Light seeks vengeance

   Items 2 & 3 seem like they would often be connected: if Light is to be a good character, what reason does he have to use a Death Note? Vengeance is one of the few socially permissible uses. Of course, Light could start as a good character using the Death Note for vengeance and slide down to an evil ending, but it’s not as likely.

4. Light seeking vengeance for a friend rather than his mother

   This item is contradictory, but only weakly so: a switch between mother and friend is an easy change to make, one which doesn’t much affect the rest of the plot.

On net, these 4 items clearly favor the hypothesis of the script being real. But how much? How much would we expect the fan or faker to avoid Hollywood-style changes compared to actual Hollywood screenwriters like the Parlapanides?

This is the exact same question we already considered in the plot section of internal evidence! Now that we have external attestation that some of the plot changes I identified back in 2009_15ya as being Hollywood-style are in the real script, can we do calculations?

I don’t think we can. The external attestation proves I was right in fingering those plot changes as Hollywood-style, but this is essentially a massive increase in $P\left(b|a\right)$ (the chance a real script will have Hollywood-style changes is now ~100%)… but what we didn’t know before, and still do not know now, is the other half of the problem, $P\left(b|\neg a\right)$ (the chance a fake script will have similar Hollywood-style changes).

We could assume that a fake script has 50% chance of making each change and item 4 negates one of the others (even though it’s really weaker), for a total likelihood ratio of $\frac{1.0}{0.5}+\frac{1.0}{0.5}+\frac{1.0}{0.5}-\frac{0.5}{1.0}=5.5$, but like before, we have no real ground to defend the 50% guess and so we will be conservative and drop this argument like its sibling argument.

Legal Takedowns

Our observation here is that the two sendspace links hosting the PDF went dead within days of the Anime Vice post. We don’t know for sure that the links went dead due to takedown, and we don’t know for sure that a takedown would be sent only if the script was real. These uncertainties transform what seems like a slam dunk proof (“a takedown would be done only if the studio complained, and the studio would complain only if it was real! A takedown was down, therefore it was real!”) into just another probabilistic question for us.

Do people send fake takedowns to file hosts? In my experience, a few people (based on watching people upload music and manga etc.) attract trolls dedicated to sending complaints about anything they post, and sometimes entire sites lose their file hosting (eg. in May 2012_12ya, MikuDB lost hosting for >1000 Vocaloid & doujin music albums) but in general downloads work for years afterwards. An instructive sample is to look at the most recent MediaFire-related submissions to Reddit (from the last 8 days when I checked) and see how many of the first 25 are dead for copyright-related reasons; when I tried, only 1 was blocked over copyright¹⁰, giving a 4% takedown rate. I regard a fake takedown within days as a remote chance, but let’s call it 5%.

Do studios send takedowns for fanfiction? No, essentially never: it’s a big thing when an author like Anne Rice chooses to crack on fanfiction, or when J.K. Rowling sues the publisher of a fan-work. The 24,246 DN fanfics on FanFiction.net stand testimony to the disinclination of studios and publishers to crack down. This chance might as well be zero, but we’ll call it 5% anyway for symmetry.

Do studios send takedowns for real scripts? Yes, frequently (much to the disgust of script collectors). The previously-mentioned TMNT script leak seems to have been partially suppressed with DMCA takedowns. Isn’t it quite plausible that this is what happened? But let’s call it just 50% as usual. Maybe plenty of real scripts get posted to news sites and the big studio like Warner Brothers does absolutely nothing about it.

Once we have settled on 5%/5%/50%, it’s as routine as usual to work out the new posterior:

1. a = is real

2. b = copies get taken down

3. $P\left(a\right)$ = probability of being real = 0.942

4. $P\left(\neg a\right)$ = probability of being not real = 1 - 0.942 = 0.05149

5. $P\left(b|a\right)$ = probability a real script will have copies taken down = 50% = 0.50

6. $P\left(b|\neg a\right)$ = probability a fake script will have copies taken down = 5%+5% = 0.10

Then Bayes:

$$P\left(a|b\right)=\frac{P\left(b|a\right)\cdot P\left(a\right)}{\left(P\right(b|a)\cdot P(a\left)\right)+\left(P\right(b|\neg a)\cdot P(\neg a\left)\right)}=\frac{0.5\cdot 0.942}{(0.5\cdot 0.942)+(0.1\cdot 0.058)}=0.9878$$

Likelihood ratio:

$$\frac{0.50}{0.10}=5$$