Preface: I wish to acknowledge that this article was written with full reference to http://www.adobe.com/devnet/acrobat/pdfs/PDF32000_2008.pdf. Most of all that I have learned about PDFs are from the above reference. If you are really interested take time to read it. Surprisingly, it is easy and interesting to read! I am writing this tutorial out of my interest in knowing the PDF specification. My quest started when I tried hard but failed to extract text from a simple PDF file that contained a single page of text. Please let me know (steve@printmyfolders.com) if you find any errors. I have relied on the PDF specification (link on page top) to create this tutorial. This tutorial covers PDF files conforming to the ISO 32000-1 specification (Pages vi to viii in the PDF32000_2008.pdf give more information on this).

 

"At the core of PDF is an advanced imaging model derived from the PostScript® page description language. This PDF Imaging Model enables the description of text and graphics in a device-independent and resolution-independent manner. To improve performance for interactive viewing, PDF defines a more structured format than that used by most PostScript language programs. Unlike Postscript, which is a programming language, PDF is based on a structured binary file format that is optimized for high performance in interactive viewing. PDF also includes objects, such as annotations and hypertext links, that are not part of the page content itself but are useful for interactive viewing and document interchange." (Quoted from Page vii of PDF32000_2008.pdf).

White-space characters: Null, Horizontal Tab, Line Feed, Form feed, Carriage return and space. White-space characters separate names and other objects from each other. Interestingly, PDF treats all white-space characters outside a comment, string or stream the same. Outside a comment, string or stream PDF considers any sequence of consecutive white-space characters as one character. What this means is that you may have 5 spaces but in reality it is considered as one. Note that this does not apply to white-space characters within strings, streams and comments. The Carriage return and Line feed are considered as end-of-line markers (EOL). Carriage return followed immediately by a Line feed is considered as one EOL marker.

 

Delimiter characters: (, ), <, >, [, ], {, }, / and % (4 pairs and 2 unique). These are used in the objects which we would look at later. They basically delimit (mark the boundary or border) for entities.

 

Regular character: All characters other than White-space and Delimiter characters including those that are not part of the standard ASCII character set.

An interesting fact to note is that a PDF can consist entirely of just ASCII characters or can consist of ASCII characters and binary data (that are outside the group of ASCII characters). Most PDf files that are encrypted or contain images will have binary data (images are represented in binary). PDF files that contain binary data get corrupted when edited or even opened and saved in normal text editors like Notepad. 

 

You may also wonder why you don't see any text or its equivalent when opening a PDF file in a text editor or even binary editor. There may be two reasons. The first and most common reason is that the content stream (where the text is stored/kept) is encoded (transformed/changed) to conserve space. This is what happens with most files.The second reason could be that the PDF file is encrypted purposely to keep the text secure.

2. Numeric values: There are two types; integer & real. Integers are numbers without any decimal points and can have a + or – symbol preceding it. For instance the number 10 is an example of an integer. Real numbers must have a decimal point. 10.5, 0.0, +5.0, -1.0. Real numbers cannot be expressed in exponential format. 

3. Strings: Strings contain characters (can be zero characters as well). They can be literal characters within parenthesis or hexadecimal data within angle brackets. Notice that the parenthesis and angle brackets are delimiter characters that we learnt earlier. 

(I love Java (and PDF))

There are escape characters that can be used. Refer to the PDF specification for more details. The sequence \ddd where ddd is an octal character can be used to represent characters outside the ASCII character set.

Here is an example of a String represented with hexadecimal numbers. 

<48454C4C4F>

<4845C> will be considered as <4845C0>

4. Names: Names consist of a sequence of characters (except null). A forward slash / must be used to introduce the name. In case hash (#) is part of the name, use # followed by its hexadecimal code 23. To represent characters using their hexadecimal value use # followed by the hexadecimal value. All characters that are not regular characters have to be represented by the # followed by their hexadecimal value. Please refer to the PDF specification for more details.

/MyName represents MyName

/My#20Name represents My Name

/All#20#23Numbers represents All #Numbers

5. Arrays: These are similar to the arrays found in computer languages but these arrays are different in that they can contain different object types (including strings, names and even other arrays). Arrays are represented within square brackets. 

6. Dictionaries:

<< /Name (Steve)

/Age 99

/Address << /Number 1234

                /Street (Java Street)

                /Suburb (Java Town)

                /PostCode (4321)

                >>

>>  

7. Streams: Streams represent large data that cannot fit into a String. An image, for instance, can be represented as a stream. As you will see later, the contents of each page in PDF is represented as a 'Contents' stream. It consists of a dictionary followed by the keyword ‘stream’, newline, the stream’s data and the keyword ‘endstream’.

stream

……….

endstream

The dictionary contains the length (number of bytes) of the stream. This must have a ‘Length’ entry to mention the length (number of bytes) of the stream. An error occurs if the stream has more bytes of data that the length mentioned in the dictionary. As of PDF 1.2, the stream’s data can be in an external file and the dictionary will hold the details of the external file. In such a case, the data between the keywords ‘stream’ and ’endstream’ is ignored (by the reader). There are other optional entries for the stream dictionary but discussing these are beyond the scope of this tutorial.

Having looked at the basic objects, let's go further and look at other components of a PDF file.

Indirect Objects

The identifier has two parts. The first part is the object number that can be any positive integer. The second part is a generation number – the first ‘version’ of the PDF file gets 0. The updated version gets a higher number and so on. You declare an indirect object within the keywords ‘obj’ and ’endobj’. Be aware that the combination of the object number and generation number has to be unique. 

1 0 obj

(my biography)

endobj

The object number is 1 and the generation number is 0. The indirect object is the string object (my biography).  If another object wanted to refer to this object, it would quote the object number, followed by space, generation number, followed by space and then R.

1 0 R

It is not an error if one was to refer to a non existing indirect object.

File Structure: A PDF file has the following structure

1.       A Header (not more than a line) showing the PDF specification this file follows.

2.       A File Body that has the objects of the file

3.       A Cross-reference table that has info about indirect objects

4.       A Trailer that shows the location of the cross-reference table and special objects within the body of the file.

File Header: This denotes the PDF specification version of the PDF file. %PDF- followed by version number 1.N, where N is a digit between 0 & 7.

%PDF-1.2

As mentioned earlier this is actually a comment that is used to specify the PDF version.

Beginning with version 1.4 the document's catalog dictionary is used instead of this. If the file has binary data, there will be at least four binary characters, immediately after the header. This is to show PDF reading applications (like Adobe reader) that the PDF has binary data. Again, when opening some PDF files in their raw form (as in a text editor) you may notice the four binary characters just after the first comment.

File Body: The File body consists of indirect objects (discussed earlier). These objects represent text and other details (like font type etc.) used in displaying PDF.  As of version 1.5, the body can also contain object streams. 

Cross-Reference Table: This table is similar to a directory. It contains the location of each object within the PDF file. By looking at the entries in this table, the PDF reading application (for example, Adobe Reader), can easily locate an object within the file. This saves time as the object is accessed in a random manner (rather than reading every line of the file). The cross-reference table can have one or more sections. Each of these sections can have one or more subsections. 

Each section begins wth the word 'xref'. Following this line are two numbers separated by a single space. The first number is the object number of the first of the series of objects listed below it. The second number refers to the number of entries in that subsection. For a PDF file that has been created for the first time or a PDF file that has not been incrementally updated, there shall be only one subsection and the object numbering starts with 0. Notice that the object numbers have to be consecutive. In the example below, it is safe to assume that entries for objects 1, 2, 3 & 4 will follow. 

xref

0 5  

Following this are the entries for each object. Each entry shall be exactly 20 bytes long. The entries are of the format

nnnnnnnnnn ggggg meol

nnnnnnnnnn - This is a ten digit value. This reveals how far the object is from the start of the file. For instance, the value 100 denotes that the object is 100 bytes from the start of the file. 

ggggg - 5-digit generation number

m - can be either 'n' or 'f'. 'n' denotes that the object is still in use and 'f' denotes that the object has been deleted and is free. 

eol - end of line. Consists of 2 chars.

The ten digits, followed by space, followed by five digits, followed by space, followed by a single character and the eol make exactly 20 digits. If the first two numbers are not long enough, to be ten and five digits respectively, zeroes are added to the front. 

Let's come back to the 0 5 that we saw in the example earlier.The 0 denotes the object number of the first object in this subsection. The value 5 denotes that there are 5 entries (including the one for 0) and that the remaining four entries are for objects with object numbers 1, 2, 3 and 4. The first entry at the cross-reference table is for object 0. Object 0 will have 0000000000 as its first ten digits (if there are no other free objects) and will always have 65535 as its 5-digit generation number. It also shall have 'f' as the character.  

xref

0 2

0000000000 65535 f

...........

In the above case, in addition to objects 0, 1, 2, 3 there are two other objects 9 & 10.

 

An object cannot be entered in more than one subsection within a section.

 

When an indirect object is deleted, its entry is marked as free (by changing the n to f) and linked to the linked list of free objects. It's generation number is increased by 1. The object's generation number gets updated each time the object gets deleted and can go upto a maximum of 65,535. For instance, an indirect object that was referenced as 1 0 will become 1 1 when reused.