Cross Site Scripting

Welcome to the hackers playground.

Cross-Site Scripting attacks are a type of injection problem, in which malicious scripts are injected into the otherwise benign and trusted web sites. Cross-site scripting (XSS) attacks occur when an attacker uses a web application to send malicious code, generally in the form of a browser side script, to a different end user.

An attacker can use XSS to send a malicious script to an unsuspecting user. The end user’s browser has no way to know that the script should not be trusted, and will execute the script. Because it thinks the script came from a trusted source, the malicious script can access any cookies, session tokens, or other sensitive information retained by your browser and used with that site. These scripts can even rewrite the content of the HTML page.

Most Web sites today add dynamic content to a Web page making the experience for the user more enjoyable. Dynamic content is content generated by some server process, which when delivered can behave and display differently to the user depending upon their settings and needs. Dynamic Web sites have a threat that static Web sites don’t, called “cross-site scripting,” also known as “XSS.”

“A Web page contains both text and HTML markup that is generated by the server and interpreted by the client browser. Web sites that generate only static pages are able to have full control over how the browser user interprets these pages. Web sites that generate dynamic pages do not have complete control over how their outputs are interpreted by the client. The heart of the issue is that if untrusted content can be introduced into a dynamic page, neither the Web sites nor the client has enough information to recognize that this has happened and take protective actions,” according to CERT Coordination Center, a federally funded research and development center to study Internet security vulnerabilities and provide incident response.

Cross-site scripting is gaining popularity among attackers as an easy exposure to find in Web sites. Every month cross-site scripting attacks are found in commercial sites and advisories are published explaining the threat. Left unattended, your Web site’s ability to operate securely, as well as your company’s reputation, may become victim of the attacks.

This article is written to raise the awareness of this emerging threat and to present a solution implementation for Web applications to avoid this kind of attack.

The threats of cross-site scripting

Cross-site scripting poses server application risks that include, but are not limited to, the following:

  • Users can unknowingly execute malicious scripts when viewing dynamically generated pages based on content provided by an attacker.
  • An attacker can take over the user session before the user’s session cookie expires.
  • An attacker can connect users to a malicious server of the attacker’s choice.
  • An attacker who can convince a user to access a URL supplied by the attacker could cause script or HTML of the attacker’s choice to be executed in the user’s browser. Using this technique, an attacker can take actions with the privileges of the user who accessed the URL, such as issuing queries on the underlying SQL databases and viewing the results and to exploit the known faulty implementations on the target system.

 

Launching an attack

After an application on a Web site is known to be vulnerable to cross-site scripting, an attacker can formulate an attack. The technique most often used by attackers is to inject JavaScript, VBScript, ActiveX, HTML, or Flash for execution on a victim’s system with the victim’s privileges. Once an attack is activated, everything from account hijacking, changing of user settings, cookie theft and poisoning, or false advertising is possible.

 

Sample attack scenarios

The following scenario diagrams illustrate some of the more relevant attacks. We will not, however, be able to list all variants of the vulnerability. To learn more about the documented attacks and how to protect yourself as a vendor or as a user, see the Resources section.

Scripting via a malicious link

In this scenario, the attacker sends a specially crafted e-mail message to a victim containing malicious link scripting such as one shown below:

<A HREF=http://legitimateSite.com/registration.cgi?clientprofile=<SCRIPT>malicious code</SCRIPT>>Click here</A>

When an unsuspecting user clicks on this link, the URL is sent to legitimateSite.com including the malicious code. If the legitimate server sends a page back to the user including the value of clientprofile, the malicious code will be executed on the client Web browser

 

Stealing users’ cookies

If any part of the Web site uses cookies, then it may be possible to steal them from its users. In this scenario, the attacker files a page with malicious script to the part of the site that is vulnerable. When the page is displayed, the malicious script runs, collects the users’ cookies, and sends a request to the attacker’s Web site with the cookies gathered. Using this technique, the attacker can gain sensitive data such as passwords, credit card numbers, and any arbitrary information the user inputs

Sending an unauthorized request

In this scenario, the user unknowingly executes scripts written by an attacker when they follow a malicious link in a mail message. Because the malicious scripts are executed in a context that appears to have originated from the legitimate server, the attacker has full access to the document retrieved and may send data contained in the page back to their site.
If the embedded script code has additional interactions capability with the legitimate server without alerting the victim, the attacker could develop and exploit that posted data to a different page on the legitimate Web server

Avoiding an attack

As stated above, cross-site scripting is achieved when an attacker is able to cause a legitimate Web server to send a page to a victim user’s Web browser that contains a malicious script of the attacker’s choosing. The attacker then has the malicious script run with the privileges of a legitimate script originating from the legitimate Web server.

Now that we know the basis of an attack, what can we do to protect ourselves from this vulnerability?

Web site developers can protect their sites from being abused in conjunction with these attacks by ensuring that dynamically generated pages do not contained undesired tags.

From the Web user’s perspective, two options exist to reduce the risk of being attacked through this vulnerability. The first — disabling scripting languages in the Web browser as well as the HTML-enabled e-mail client — provides the most protection but has the side effect of disabling functionality. The second — only following links from the main Web site for viewing — will significantly reduce a user’s exposure while still maintaining functionality.

However, none of the solutions that Web users can take are complete solutions. In the end, it is up to Web page developers to modify their pages to eliminate these types of problems. This can be accomplished by properly filtering and validating the input received and properly encoding or filtering the output returned to the user.

Filtering

The basis of this approach is never trust user input and always filter metacharacters (“special” characters) that are defined in the HTML specification. Each input field, including link parameters will be validated for script tags. When found and dependent on the context, the input will be rejected and thus prevent the malicious HTML from being presented to the user.

Adding to the complexity is that many Web browsers try to correct common errors in HTML. As a result, they sometimes treat characters as special when, according to the specification, they are not. Therefore, it is important to note that individual situations may warrant including additional characters in the list of special characters. Web developers must examine their applications and determine which characters can affect their web applications.

Filtering on the input side is less effective because dynamic content can be entered into a Web site database via methods other than HTTP. In this case, the Web server may never see the data as part of the data input process and the data elements still remain tainted. Alternatively, it is recommended that filtering be done as part of the data output process, just before it is rendered as part of the dynamic page. Done correctly, this approach ensures that all dynamic content is filtered.

Encoding

Cross-site scripting attacks can be avoided when a Web server adequately ensures that generated pages are properly encoded to prevent unintended execution of scripts.

Each character in the ISO-8859-1 specification can be encoded using its numeric entry value. Server side encoding is a process where all dynamic content will go through an encoding function where scripting tags will be replaced with codes in the chosen character set.

Generally speaking, encoding is recommended because it does not require you to make a decision about what characters could legitimately be entered and need to be passed through. Unfortunately, encoding all untrusted data can be resource intensive and may have a performance impact on some Web servers.

 

Which strategy is right for me?

CGI-based Web applications or applications that favor field edit check at the browser will likely adapt to the filtering strategy by extending the existing field edit check to cover the cross-site scripting vulnerability. Note that although browser side field edit check saves a few runs back to the server, it only works for the honest user and requires thorough code walkthroughs to guarantee that all input fields are checked in order to meet the remediation recommendation. Web applications with server side validation designed-in, however, can have a choice to adapt to either or both strategies.

For the filtering strategy to work properly, Web developers need to ensure that the list of metacharacters for filtering is up-to-date according to the needs of their applications. The encoding strategy, on the other hand, does not have the above-described maintenance effort, and it also has less impact on the existing application code as well as on application functionality. For these reasons, the encoding strategy appears to be a favorite choice of implementation. A sample encoding implementation is described next.

 

The 1-2-3 of a sample encoding

A simple, yet effective, way for a Web server to ensure that the generated pages are properly encoded is to pass each character in the dynamic content through an encoding function where the scripting tags in the dynamic content are replaced with codes in the chosen character set. This task is perfect for a custom tag library.

Custom tag library basics

A custom tag library is comprised of one or more Java language classes (called tag handlers) and an XML tag library description file (TLD), which dictates the new tag names and valid attributes for those tags. Tag handlers and TLDs determine how the tags, their attributes, and their bodies will be interpreted and processed at request time from inside a JSP page. A custom tag library provides an architecture that is more flexible than a Java bean at encapsulating a complex operation.

 

Our “custom” fitted tag library

What better name is there for our custom tag library besides naming it XSS? Tag libraries are software components that are plugged into a servlet container. The servlet container creates tag handlers, initializes them and calls the doStartTag(), doEndTag() and release() methods, in that order.

Through these interactions, our XSS custom tag library will be able to apply the “custom” action of encoding the dynamic data found on a JSP page. Implementing custom tags is straightforward and the steps are as follows:

  • Create a tag library descriptor (.tld) describing the tags.
  • Add a taglib directive to JSP files that use the tags.
  • Implement a tag handler that extends TagSupport and overrides the doStartTag() or the doEndTag() methods.

TLD (tag library descriptor)

A tag library descriptor is an XML file whose elements describe a particular tag library. The tld file for our XSS custom tag library, is shown in Listing 1. The tag element defines the encode action, including an attribute, property. The tagclass element defines the tag handler class EncodeTag.
Listing 1. The xss.tld file

				
<?xml version="1.0" encoding="UTF-8"?>
 DOCTYPE taglib 
   PUBLIC "-//Sun Microsystems, Inc.//DTD JSP Tag Library 1.1//EN" 
   "http://java.sun.com/j2ee/dtds/web-jsptaglibrary_1_1.dtd">

<taglib>
     <tlibversion>1.0</tlibversion>
     <jspversion>1.1</jspversion> 
     <tag>
        <name>encode</name>
        <tagclass>dw.demo.xss.EncodeTag</tagclass>
        <bodycontent>empty</bodycontent>
        <attribute>
            <name>property</name>
            <required>true</required> 
        </attribute>
     </tag>
<taglib>

 

The taglib directive

The taglib directive identifies the tag library descriptor and defines the tag prefix that associates subsequent tags with the library. A sample taglib directive, which appears in the JSP using the XSS custom tag library, is shown below:

 

			
   <%@ taglib uri="/WEB-INF/tlds/xss.tld" prefix="xss" %>

 

Coding the tag handler

A tag handler is an object in the Web container that helps evaluate actions when a JSP page executes. The EncodeTag class is the tag handler for the encode action. Its doStartTag method, which encodes the dynamic content to the ISO-8859-1 character set, is shown in Listing 2.
Listing 2. Encoding the dynamic content

				
 public int doStartTag() throws JspException {

     StringBuffer sbuf = new StringBuffer();

     char[] chars = property.toCharArray();
     for (int i = 0; i < chars.length; i++) { 
          sbuf.append("&#" + (int) chars[i]);
     }     

     try{
          pageContext.getOut().print(sbuf.toString());     
     } catch (IOException ex) {
          throw new JspException(ex.getMessage());     
     }     

     return SKIP_BODY;
 }

 

 

Deployment

The XSS custom tag library, which is part of a Web application, is packaged as additional files into the Web application’s WAR file as follows:

  • WEB-INF/lib/encodetaglib.jar
  • WEB-INF/tlds/xss.tld

 

Putting it to work

The following scenario illustrates how the custom tag library would be used. Suppose that a hypothetical Web site for receiving articles included a page for reviewing articles that you have subscribed-to. The dynamic content, article items intended for you, is prepared inside a JSP file using the <%= expression %> syntax.

Advertisements

What is Ethical Hacking

By P. Walsh
An Ethical Hacker is an expert hired by a company to attempt to attack their network and computer system the same way a hacker would. Ethical Hackers use the same techniques and tactics as those used by illegal hackers to breach corporate security systems. The end result is the company’s ability to prevent an intrusion before it ever occurs.

A company can’t know if their security system is solid unless they test it. It’s hard, though, for a company’s IT team to thoroughly ring out the system. Try as they might, the techs can’t go at the system with all the malicious or mischievous motives of a true illegal hacker. To thoroughly uncover vulnerabilities, the theory goes; you must examine your security system through the eyes of an illegal hacker.

The word hacking has strongly negative connotations, and, for the most part, rightly so. But ethical hacking is much different. It takes place with the explicit permission of the company whose system is being attacked. In fact, their “good guy” role is underscored by the nickname “white hat” Ethical Hackers have been given. The nickname is a throwback to old Westerns where the good cowboys could be identified by their white hats.

The company and the Ethical Hacker enter into a legally binding contract. The contract, sometimes called a “get out of jail free card,” sets forth the parameters of the testing. It’s called the “get out of jail free card” because it’s what harbors the Ethical Hacker from prosecution. Hacking is a felony, and a serious one at that. The terms of the agreement are what transform illegal behavior into a legal and legitimate occupation.

Once the hacker has exhausted his attempts, he reports back to the company with a list of the vulnerabilities he uncovered. The list in and of itself, however, is not particularly useful. What’s most valuable is the instructions for eliminating the vulnerabilities that the Ethical Hacker provides.

An Ethical Hacker works to uncover three key pieces of information. First, he determines what information an illegal hacker can gain access to. Next, he explores what an illegal hacker could do with that information once gained. Last, the Ethical Hacker ascertains whether an employee or staff member would be alerted to the break-in, successful or not.

At first it might sound strange that a company would pay someone to try to break into their system. Ethical hacking, though, makes a lot of sense, and it is a concept companies have been employing for years. To test the effectiveness and quality of product, we subject it to the worst case scenario. The safety testing performed by car manufacturers is a good example. Current regulatory requirements including HIPAA, Sarbanes Oxley, and SB-1386 and BS 799 require a trusted third party to check that systems are secure.

In order to get the most out of the assessment, a company should decide in advance the nature of the vulnerabilities they’re most concerned with. Specifically, the company should determine which information they want to keep protected and what they’re concerned would happen if the information was retrieved by an illegal hacker.

Companies should thoroughly assess the qualifications and background of any Ethical Hacker they are considering hiring. This individual will be privy to highly sensitive information. Total honesty and integrity is of the utmost importance.

How to Become an Ethical Hacker

Not all hackers do evil work. Here’s what you need to know to use your hacking skills to do good.

Do viruses, DDoS attacks, or buffer overflows tickle your fancy? If so, you might consider becoming a legal hacker, aka an ethical hacker, “white hat” hacker, or penetration tester.

Businesses and government-related organizations that are serious about their network security hire ethical hackers and penetration testers to help probe and improve their networks, applications, and other computer systems with the ultimate goal of preventing data theft and fraud. You may not get the same adrenaline rush that you might with underground hacking, but you can earn a good and honest living–and not end up facing prison time, as some illegal “black hat” hackers do.

How does the job market look like for ethical hackers? Extremely good! The IT market overall continues to grow despite the current economic turmoil. Research firm Gartner estimates that worldwide enterprise IT spending grew by 5.9 percent between 2009 and 2010, to a total of $2.7 trillion. At the same time, security is becoming a more pressing concern. Gartner expects to see an increase of nearly 40 percent in spending on worldwide security services during the five-year period from 2011 to 2015, eventually surpassing $49.1 billion.

In your first years as an ethical hacker, you’ll be in a position to earn anywhere from $50,000 to $100,000 per year, depending on the company that hires you, and on your IT experience and education. With several years of professional experience, you could command $120,000 or more per year, especially if you do your own independent consulting.

You can’t just dive into an ethical hacker position, however. Without IT security experience, you won’t get very far, even with degrees and certifications. As is true for other IT jobs, employers typically want candidates who have college degrees, but related experience is king. And experience with certifications can typically take the place of some degree requirements.

Getting Started

What you need to do to get started on the road to becoming an ethical hacker depends on where you are in the IT field. If you haven’t started your IT career yet, you might even consider military service. The military offers many IT opportunities, and you get paid to go to school, even if you enlist in a part-time branch such as the National Guard or Reserves. Military service also looks good to employers that require security clearances.

Start with the basics: Earn your A+ Certification and get a tech support position. After some experience and additional certification (Network+ or CCNA), move up to a network support or admin role, and then to network engineer after a few years. Next, put some time into earning security certifications (Security+, CISSP, or TICSA) and find an information security position. While you’re there, try to concentrate on penetration testing–and get some experience with the tools of the trade. Then work toward the Certified Ethical Hacker (CEH) certification offered by the International Council of Electronic Commerce Consultants (EC-Council for short). At that point, you can start marketing yourself as an ethical hacker.

For a hacker, networking know-how is vital; but make sure that you gain experience in related areas as well. Discover and play with Unix/Linux commands and distributions. Make sure you also learn some programming–maybe C, LISP, Perl, or Java. And spend some time with databases such as SQL.

Soft Skills

Hacking isn’t all technical. It also requires so-called soft skills, just as any other IT job does. You’ll need a strong work ethic, very good problem-solving and communications skills, and the ability to say motivated and dedicated.

Ethical hackers also need street smarts, people skills, and even some talent for manipulation, since at times they need to be able to persuade others to disclose credentials, restart or shut down systems, execute files, or otherwise knowingly or unknowingly help them achieve their ultimate goal. You’ll need to master this aspect of the job, which people in the business sometimes call “social engineering,” to become a well-rounded ethical hacker.

Stay Legal!

It’s important never to engage in “black hat” hacking–that is, intruding or attacking anyone’s network without their full permission. Engaging in illegal activities, even if it doesn’t lead to a conviction, will likely kill your ethical hacking career. Many of the available jobs are with government-related organizations and require security clearances and polygraph testing. Even regular companies will perform at least a basic background check.

Becoming a Certified Ethical Hacker (CEH)

As noted earlier, becoming a Certified Ethical Hacker (CEH) involves earning the appropriate credential from the EC-Council after a few years of security-related IT experience. The certification will help you understand security from the mindset of a hacker. You’ll learn the common types of exploits, vulnerabilities, and countermeasures.

Qualification for a CEH (a vendor-neutral certification) involves mastering penetration testing, footprinting and reconnaissance, and social engineering. The course of study covers creating Trojan horses, backdoors, viruses, and worms. It also covers denial of service (DoS) attacks, SQL injection, buffer overflow, session hijacking, and system hacking. You’ll discover how to hijack Web servers and Web applications. You’ll also find out how to scan and sniff networks, crack wireless encryption, and evade IDSs, firewalls, and honeypots.

Through approved EC-Council training partners, you can take a live, five-day onsite or online training course to prepare for the CEH cert. You can generally take live online classes over five consecutive days; onsite courses typically offer the content spread over a couple weeks for locals. In addition, you can take self-paced courses and work with self-study materials (including the CEH Certified Ethical Hacker Study Guide book) with or without the training courses. The EC-Council also offers iLabs, a subscription based-service that allows you to log on to virtualized remote machines to perform exercises.

The EC-Council usually requires that you have at least two years of information-security-related work experience (endorsed by your employer) in addition to passing the exam before it will award you the official CEH certification.

Resources

If you’re interested in ethical hacking, you can consult many useful resources for more information. To start, check the resources section of the EC-Council site. A quick Amazon search will reveal many books on ethical hacking and the CEH certification, as well.

With some googling, you can find simple hacking how-tos, which may motivate you even more. Consider downloading the Firefox add-on Firesheep or the Android app Droidsheep, and hijack your online accounts via Wi-Fi (but don’t use these tools to hijack others’ accounts–you could find yourself in legal trouble if you do).

Another option is to experiment with the BackTrack live CD. Try enabling WEP security on your wireless router at home, and then take a stab at cracking it. Check out Hack This Site to test and expand your skills. You could even set up a Linux box with Apache or buy a used Cisco router and see what you can do with it. If you want to play with malware, consider downloading–cautiously, and at your own risk–a malware DIY kit or a keylogger, and use it to experiment on a separate old PC or virtual machine.

Like other IT areas, hacking has conventions and conferences dedicated to it, such as DefCon, one of the oldest and largest of these. Such gatherings can be a great place to meet and network with peers and employers, and to discover more about hacking. DefCon also has affiliated local groups in select areas.

And remember, never attack or intrude on anyone else’s network or computers without full written permission.

Eric Geier is the founder of NoWiresSecurity, which helps businesses easily protect their Wi-Fi networks with the Enterprise mode of WPA/WPA2 security by offering a hosted RADIUS/802.1X service. He is also a freelance tech writer—become a Twitter follower or use the RSS Feed to keep up with his writings

Pro’s and Con’s of Ethical Hacking

The use of ethical hackers to test for security vulnerabilities is as old as the IT hills. But, unless there are clear goals outlining why and to what extent your organization is engaging them, the outcome could be useless information — or worse.

On the surface, ethical hacking sounds like a pretty straightforward process: You hire somebody to break into your network or application or Web servers, and report what they find. But this simple description, which does adequately explain the basic principal, masks a process that requires a great deal more thought.

Unless you first know what it is you are looking for and why you are hiring an outside vendor to hack your systems in the first place, chances are you won’t get much out of the experience, said Arian Evan, a senior security engineer at FishNet Security. Sure, you will find out your network needs to be patched or there are X number of security holes, but if that information is not relatable back to the business in some form, it’s pretty much useless.

“If you just want numbers, any of us can run a scan and give you results,” agreed Paul Klahn, FishNet’s director of assessment services.

Beyond the Numbers

To get the most from a test, putting results into a business context is imperative, said Klahn. Which holes are truly a security threat? How deep into the network can a hacker get via one of these holes? Which should be patched first?

Security holes can even be a necessary part of your infrastructure, allowing you to do business with partners, for example, so closing them up for security reasons may cause more headaches than the vulnerability. Your contractor should be able to appreciate this nuance.

Invariably, threats will be found, said Albert Decker, executive director of EDS’s Security and Privacy services, and a former ethical hacker with 25 years in the business and a 99% success rate at getting around corporate security.

“It became roughly the equivalent of ‘Can you throw this brick through a window?’ and the answer is, invariably, unless you miss the window, it will break the glass,” Decker said, commenting on his days as a hacker.

Because not much has changed since Decker was actually scanning code, the firm you hire should be able to provide you with a threat assessment and articulate remedies that take into account business needs. And, even then, the hack should be part of a larger security audit that looks at known vulnerabilities while comparing your IT governance policies and procedures against industry best practices.

Snapshot

The reason for this, said Jim Goddard, an ethical hacker at IBM, is simple: If you just hire a hack and do nothing else, on the day it’s complete, you are no more secure than the day before the hack began. This is because hacking provides just a snapshot of your overall security. Yes, it can provide confirmation your security is good or expose unknown threats, but it can’t tell you what those threats will be tomorrow. One configuration change and much of the hacker’s work can be negated, agreed Decker.

“The use of hackers is essentially a point-in-time test for a continuous problem,” said Decker. “It’s only giving you one very narrow slice of your environment which could change, literally, the second after the test is completed.”

There are four basic kinds of hacks you can have done, said Goddard:

 

  • IP Hack: You hire someone to hack a specific IP address, giving them little or no information beforehand (Be careful if the IP address is an overseas server. You don’t want hackers hacking the wrong IP address, like a foreign government’s computers, causing an international incident.);

 

  • Application Hack: A much more sophisticated hack that can delve deep into databases and down production servers. Only experienced hackers, with strict guidelines governing their actions, should be allowed to perform such tests. Never hire a “reformed” black-hat hacker for this type of test;

 

  • Physical Infrastructure Hack: This is where people try to get into your facilities to access your systems or go dumpster diving looking for confidential information such as passwords discarded on sticky notes; and

 

  • Wireless Hack: War-driving is the new term to describe this type of attack where wireless access points are exploited from the back of a van. Ethical hackers do the same thing, but report their findings back to you instead of stealing your passwords. Have them check out your teleworkers as well to see if home offices are a source of entry to your network.

For any of these tests, a reputable firm with clearly defined methodologies should be hired, cautioned Goddard. If a company can’t tell you exactly how it conducts its business, move on. And never hire former hackers to do the work on the cheap. They may not be as reformed as they say and could leave back doors behind or worse, he said.

Scope & Limits

Once a vendor is selected (never use the RFP process for this type of work, cautions Evans, interview prospective companies), it is very important to outline and define the scope of the project — you don’t want the hacker deciding where to start and stop an attack. Delegate a point person with decision-making authority the hackers can contact day or night if problems arises and authority to continue is required.

But, perhaps most importantly, know what you are looking to get from the experience. Too often, said Decker, companies conduct these tests and feel they are secure. This is not the case. Ethical hacking is just another tool, not a panacea. If viewed as such, it will fall into its proper place alongside other security tools. If not, it can leave you far more exposed through either false feelings of security or outright damage to your systems.

“There’s many, many different things we can do on a network that fall in or around ‘ethical’ hacking,” said FishNet’s Evans, ” … but, without that business case, its very hard to help the client make decisions about what technology services and perspectives they need.”

Cryptography

An Overview of Cryptography

Gary C. Kessler
22 May 2011

A much shorter, edited version of this paper appears in the 1999 Edition of Handbook on Local Area Networks, published by Auerbach in September 1998. Since that time, this paper has taken on a life of its own…


CONTENTS

1. INTRODUCTION
2. THE PURPOSE OF CRYPTOGRAPHY
3. TYPES OF CRYPTOGRAPHIC ALGORITHMS
3.1. Secret Key Cryptography
3.2. Public-Key Cryptography
3.3. Hash Functions
3.4. Why Three Encryption Techniques?
3.5. The Significance of Key Length4. TRUST MODELS
4.1. PGP Web of Trust
4.2. Kerberos
4.3. Public Key Certificates and Certification Authorities
4.4. Summary
5. CRYPTOGRAPHIC ALGORITHMS IN ACTION
5.1. Password Protection
5.2. Some of the Finer Details of Diffie-Hellman Key Exchange
5.3. Some of the Finer Details of RSA Public-Key Cryptography
5.4. Some of the Finer Details of DES, Breaking DES, and DES Variants
5.5. Pretty Good Privacy (PGP)
5.6. IP Security (IPsec) Protocol
5.7. The SSL “Family” of Secure Transaction Protocols for the World Wide Web
5.8. Elliptic Curve Cryptography
5.9. The Advanced Encryption Standard and Rijndael
5.10. Cisco’s Stream Cipher
5.11. TrueCrypt
6. CONCLUSION… OF SORTS
7. REFERENCES AND FURTHER READING
A. SOME MATH NOTES
A.1. The Exclusive-OR (XOR) Function
A.2. The modulo Function
ABOUT THE AUTHOR

FIGURES

  1. Three types of cryptography: secret-key, public key, and hash function.
  2. Sample application of the three cryptographic techniques for secure communication.
  3. Kerberos architecture.
  4. GTE Cybertrust Global Root-issued certificate (Netscape Navigator).
  5. Sample entries in Unix/Linux password files.
  6. DES enciphering algorithm.
  7. A PGP signed message.
  8. A PGP encrypted message.
  9. The decrypted message.
  10. IPsec Authentication Header format.
  11. IPsec Encapsulating Security Payload format.
  12. IPsec tunnel and transport modes for AH.
  13. IPsec tunnel and transport modes for ESP.
  14. Browser encryption configuration screen (Firefox).
  15. SSL/TLS protocol handshake.
  16. Elliptic curve addition.
  17. AES pseudocode.
  18. TrueCrypt screen shot (Windows).
  19. TrueCrypt screen shot (MacOS).
  20. TrueCrypt hidden encrypted volume within an encrypted volume.

TABLES

  1. Minimum Key Lengths for Symmetric Ciphers.
  2. Contents of an X.509 V3 Certificate.
  3. Other Crypto Algorithms and Systems of Note.
  4. ECC and RSA Key Comparison.

1. INTRODUCTION

Does increased security provide comfort to paranoid people? Or does security provide some very basic protections that we are naive to believe that we don’t need? During this time when the Internet provides essential communication between tens of millions of people and is being increasingly used as a tool for commerce, security becomes a tremendously important issue to deal with.

There are many aspects to security and many applications, ranging from secure commerce and payments to private communications and protecting passwords. One essential aspect for secure communications is that of cryptography, which is the focus of this chapter. But it is important to note that while cryptography is necessary for secure communications, it is not by itself sufficient. The reader is advised, then, that the topics covered in this chapter only describe the first of many steps necessary for better security in any number of situations.

This paper has two major purposes. The first is to define some of the terms and concepts behind basic cryptographic methods, and to offer a way to compare the myriad cryptographic schemes in use today. The second is to provide some real examples of cryptography in use today.

I would like to say at the outset that this paper is very focused on terms, concepts, and schemes in current use and is not a treatise of the whole field. No mention is made here about pre-computerized crypto schemes, the difference between a substitution and transposition cipher, cryptanalysis, or other history. Interested readers should check out some of the books in the bibliography below for this detailed — and interesting! — background information.

2. THE PURPOSE OF CRYPTOGRAPHY

Cryptography is the science of writing in secret code and is an ancient art; the first documented use of cryptography in writing dates back to circa 1900 B.C. when an Egyptian scribe used non-standard hieroglyphs in an inscription. Some experts argue that cryptography appeared spontaneously sometime after writing was invented, with applications ranging from diplomatic missives to war-time battle plans. It is no surprise, then, that new forms of cryptography came soon after the widespread development of computer communications. In data and telecommunications, cryptography is necessary when communicating over any untrusted medium, which includes just about any network, particularly the Internet.

Within the context of any application-to-application communication, there are some specific security requirements, including:

  • Authentication: The process of proving one’s identity. (The primary forms of host-to-host authentication on the Internet today are name-based or address-based, both of which are notoriously weak.)
  • Privacy/confidentiality: Ensuring that no one can read the message except the intended receiver.
  • Integrity: Assuring the receiver that the received message has not been altered in any way from the original.
  • Non-repudiation: A mechanism to prove that the sender really sent this message.

Cryptography, then, not only protects data from theft or alteration, but can also be used for user authentication. There are, in general, three types of cryptographic schemes typically used to accomplish these goals: secret key (or symmetric) cryptography, public-key (or asymmetric) cryptography, and hash functions, each of which is described below. In all cases, the initial unencrypted data is referred to asplaintext. It is encrypted into ciphertext, which will in turn (usually) be decrypted into usable plaintext.

In many of the descriptions below, two communicating parties will be referred to as Alice and Bob; this is the common nomenclature in the crypto field and literature to make it easier to identify the communicating parties. If there is a third or fourth party to the communication, they will be referred to as Carol and Dave. Mallory is a malicious party, Eve is an eavesdropper, and Trent is a trusted third party.

3. TYPES OF CRYPTOGRAPHIC ALGORITHMS

There are several ways of classifying cryptographic algorithms. For purposes of this paper, they will be categorized based on the number of keys that are employed for encryption and decryption, and further defined by their application and use. The three types of algorithms that will be discussed are (Figure 1):

  • Secret Key Cryptography (SKC): Uses a single key for both encryption and decryption
  • Public Key Cryptography (PKC): Uses one key for encryption and another for decryption
  • Hash Functions: Uses a mathematical transformation to irreversibly “encrypt” information

FIGURE 1: Three types of cryptography: secret-key, public key, and hash function.

3.1. Secret Key Cryptography

With secret key cryptography, a single key is used for both encryption and decryption. As shown in Figure 1A, the sender uses the key (or some set of rules) to encrypt the plaintext and sends the ciphertext to the receiver. The receiver applies the same key (or ruleset) to decrypt the message and recover the plaintext. Because a single key is used for both functions, secret key cryptography is also calledsymmetric encryption.

With this form of cryptography, it is obvious that the key must be known to both the sender and the receiver; that, in fact, is the secret. The biggest difficulty with this approach, of course, is the distribution of the key.

Secret key cryptography schemes are generally categorized as being either stream ciphers or block ciphers. Stream ciphers operate on a single bit (byte or computer word) at a time and implement some form of feedback mechanism so that the key is constantly changing. A block cipher is so-called because the scheme encrypts one block of data at a time using the same key on each block. In general, the same plaintext block will always encrypt to the same ciphertext when using the same key in a block cipher whereas the same plaintext will encrypt to different ciphertext in a stream cipher.

Stream ciphers come in several flavors but two are worth mentioning here. Self-synchronizing stream ciphers calculate each bit in the keystream as a function of the previous n bits in the keystream. It is termed “self-synchronizing” because the decryption process can stay synchronized with the encryption process merely by knowing how far into the n-bit keystream it is. One problem is error propagation; a garbled bit in transmission will result in n garbled bits at the receiving side. Synchronous stream ciphers generate the keystream in a fashion independent of the message stream but by using the same keystream generation function at sender and receiver. While stream ciphers do not propagate transmission errors, they are, by their nature, periodic so that the keystream will eventually repeat.

Block ciphers can operate in one of several modes; the following four are the most important:

  • Electronic Codebook (ECB) mode is the simplest, most obvious application: the secret key is used to encrypt the plaintext block to form a ciphertext block. Two identical plaintext blocks, then, will always generate the same ciphertext block. Although this is the most common mode of block ciphers, it is susceptible to a variety of brute-force attacks.
  • Cipher Block Chaining (CBC) mode adds a feedback mechanism to the encryption scheme. In CBC, the plaintext is exclusively-ORed (XORed) with the previous ciphertext block prior to encryption. In this mode, two identical blocks of plaintext never encrypt to the same ciphertext.
  • Cipher Feedback (CFB) mode is a block cipher implementation as a self-synchronizing stream cipher. CFB mode allows data to be encrypted in units smaller than the block size, which might be useful in some applications such as encrypting interactive terminal input. If we were using 1-byte CFB mode, for example, each incoming character is placed into a shift register the same size as the block, encrypted, and the block transmitted. At the receiving side, the ciphertext is decrypted and the extra bits in the block (i.e., everything above and beyond the one byte) are discarded.
  • Output Feedback (OFB) mode is a block cipher implementation conceptually similar to a synchronous stream cipher. OFB prevents the same plaintext block from generating the same ciphertext block by using an internal feedback mechanism that is independent of both the plaintext and ciphertext bitstreams.

A nice overview of these different modes can be found at progressive-coding.com.

Secret key cryptography algorithms that are in use today include:

  • Data Encryption Standard (DES): The most common SKC scheme used today, DES was designed by IBM in the 1970s and adopted by the National Bureau of Standards (NBS) [now the National Institute for Standards and Technology (NIST)] in 1977 for commercial and unclassified government applications. DES is a block-cipher employing a 56-bit key that operates on 64-bit blocks. DES has a complex set of rules and transformations that were designed specifically to yield fast hardware implementations and slow software implementations, although this latter point is becoming less significant today since the speed of computer processors is several orders of magnitude faster today than twenty years ago. IBM also proposed a 112-bit key for DES, which was rejected at the time by the government; the use of 112-bit keys was considered in the 1990s, however, conversion was never seriously considered.

    DES is defined in American National Standard X3.92 and three Federal Information Processing Standards (FIPS):

    Information about vulnerabilities of DES can be obtained from the Electronic Frontier Foundation.

    Two important variants that strengthen DES are:

    • Triple-DES (3DES): A variant of DES that employs up to three 56-bit keys and makes three encryption/decryption passes over the block; 3DES is also described in FIPS 46-3 and is the recommended replacement to DES.
    • DESX: A variant devised by Ron Rivest. By combining 64 additional key bits to the plaintext prior to encryption, effectively increases the keylength to 120 bits.

    More detail about DES, 3DES, and DESX can be found below in Section 5.4.

  • Advanced Encryption Standard (AES): In 1997, NIST initiated a very public, 4-1/2 year process to develop a new secure cryptosystem for U.S. government applications. The result, the Advanced Encryption Standard, became the official successor to DES in December 2001. AES uses an SKC scheme called Rijndael, a block cipher designed by Belgian cryptographers Joan Daemen and Vincent Rijmen. The algorithm can use a variable block length and key length; the latest specification allowed any combination of keys lengths of 128, 192, or 256 bits and blocks of length 128, 192, or 256 bits. NIST initially selected Rijndael in October 2000 and formal adoption as the AES standard came in December 2001. FIPS PUB 197 describes a 128-bit block cipher employing a 128-, 192-, or 256-bit key. The AES process and Rijndael algorithm are described in more detail below in Section 5.9.
  • CAST-128/256: CAST-128, described in Request for Comments (RFC) 2144, is a DES-like substitution-permutation crypto algorithm, employing a 128-bit key operating on a 64-bit block. CAST-256 (RFC 2612) is an extension of CAST-128, using a 128-bit block size and a variable length (128, 160, 192, 224, or 256 bit) key. CAST is named for its developers, Carlisle Adams and Stafford Tavares and is available internationally. CAST-256 was one of the Round 1 algorithms in the AES process.
  • International Data Encryption Algorithm (IDEA): Secret-key cryptosystem written by Xuejia Lai and James Massey, in 1992 and patented by Ascom; a 64-bit SKC block cipher using a 128-bit key. Also available internationally.
  • Rivest Ciphers (aka Ron’s Code): Named for Ron Rivest, a series of SKC algorithms.
    • RC1: Designed on paper but never implemented.
    • RC2: A 64-bit block cipher using variable-sized keys designed to replace DES. It’s code has not been made public although many companies have licensed RC2 for use in their products. Described in RFC 2268.
    • RC3: Found to be breakable during development.
    • RC4: A stream cipher using variable-sized keys; it is widely used in commercial cryptography products, although it can only be exported using keys that are 40 bits or less in length.

       

    • RC5: A block-cipher supporting a variety of block sizes, key sizes, and number of encryption passes over the data. Described in RFC 2040.
    • RC6: An improvement over RC5, RC6 was one of the AES Round 2 algorithms.
  • Blowfish: A symmetric 64-bit block cipher invented by Bruce Schneier; optimized for 32-bit processors with large data caches, it is significantly faster than DES on a Pentium/PowerPC-class machine. Key lengths can vary from 32 to 448 bits in length. Blowfish, available freely and intended as a substitute for DES or IDEA, is in use in over 80 products.
  • Twofish: A 128-bit block cipher using 128-, 192-, or 256-bit keys. Designed to be highly secure and highly flexible, well-suited for large microprocessors, 8-bit smart card microprocessors, and dedicated hardware. Designed by a team led by Bruce Schneier and was one of the Round 2 algorithms in the AES process.
  • Camellia: A secret-key, block-cipher crypto algorithm developed jointly by Nippon Telegraph and Telephone (NTT) Corp. and Mitsubishi Electric Corporation (MEC) in 2000. Camellia has some characteristics in common with AES: a 128-bit block size, support for 128-, 192-, and 256-bit key lengths, and suitability for both software and hardware implementations on common 32-bit processors as well as 8-bit processors (e.g., smart cards, cryptographic hardware, and embedded systems). Also described in RFC 3713. Camellia’s application in IPsec is described in RFC 4312 and application in OpenPGP in RFC 5581.
  • MISTY1: Developed at Mitsubishi Electric Corp., a block cipher using a 128-bit key and 64-bit blocks, and a variable number of rounds. Designed for hardware and software implementations, and is resistant to differential and linear cryptanalysis. Described in RFC 2994.
  • Secure and Fast Encryption Routine (SAFER): Secret-key crypto scheme designed for implementation in software. Versions have been defined for 40-, 64-, and 128-bit keys.
  • KASUMI: A block cipher using a 128-bit key that is part of the Third-Generation Partnership Project (3gpp), formerly known as the Universal Mobile Telecommunications System (UMTS). KASUMI is the intended confidentiality and integrity algorithm for both message content and signaling data for emerging mobile communications systems.
  • SEED: A block cipher using 128-bit blocks and 128-bit keys. Developed by the Korea Information Security Agency (KISA) and adopted as a national standard encryption algorithm in South Korea. Also described in RFC 4269.
  • ARIA: A 128-bit block cipher employing 128-, 192-, and 256-bit keys. Developed by large group of researchers from academic institutions, research institutes, and federal agencies in South Korea in 2003, and subsequently named a national standard. Described in RFC 5794.
  • CLEFIA: Described in RFC 6114, CLEFIA is a 128-bit block cipher employing key lengths of 128, 192, and 256 bits (which is compatible with AES). The CLEFIA algorithm was first published in 2007 by Sony Corporation. CLEFIA is one of the new-generation lightweight blockcipher algorithms designed after AES, offering high performance in software and hardware as well as a lightweight implementation in hardware.
  • SMS4: SMS4 is a 128-bit block cipher using 128-bit keys and 32 rounds to process a block. Declassified in 2006, SMS4 is used in the Chinese National Standard for Wireless Local Area Network (LAN) Authentication and Privacy Infrastructure (WAPI). SMS4 had been a proposed cipher for the Institute of Electrical and Electronics Engineers (IEEE) 802.11i standard on security mechanisms for wireless LANs, but has yet to be accepted by the IEEE or International Organization for Standardization (ISO). SMS4 is described in SMS4 Encryption Algorithm for Wireless Networks (translated and typeset by Whitfield Diffie and George Ledin, 2008) or in the original Chinese.
  • Skipjack: SKC scheme proposed for Capstone. Although the details of the algorithm were never made public, Skipjack was a block cipher using an 80-bit key and 32 iteration cycles per 64-bit block.

3.2. Public-Key Cryptography

Public-key cryptography has been said to be the most significant new development in cryptography in the last 300-400 years. Modern PKC was first described publicly by Stanford University professor Martin Hellman and graduate student Whitfield Diffie in 1976. Their paper described a two-key crypto system in which two parties could engage in a secure communication over a non-secure communications channel without having to share a secret key.

PKC depends upon the existence of so-called one-way functions, or mathematical functions that are easy to computer whereas their inverse function is relatively difficult to compute. Let me give you two simple examples:

  1. Multiplication vs. factorization: Suppose I tell you that I have two numbers, 9 and 16, and that I want to calculate the product; it should take almost no time to calculate the product, 144. Suppose instead that I tell you that I have a number, 144, and I need you tell me which pair of integers I multiplied together to obtain that number. You will eventually come up with the solution but whereas calculating the product took milliseconds, factoring will take longer because you first need to find the 8 pair of integer factors and then determine which one is the correct pair.
  2. Exponentiation vs. logarithms: Suppose I tell you that I want to take the number 3 to the 6th power; again, it is easy to calculate 36=729. But if I tell you that I have the number 729 and want you to tell me the two integers that I used, x and y so that logx 729 = y, it will take you longer to find all possible solutions and select the pair that I used.

While the examples above are trivial, they do represent two of the functional pairs that are used with PKC; namely, the ease of multiplication and exponentiation versus the relative difficulty of factoring and calculating logarithms, respectively. The mathematical “trick” in PKC is to find a trap door in the one-way function so that the inverse calculation becomes easy given knowledge of some item of information.

Generic PKC employs two keys that are mathematically related although knowledge of one key does not allow someone to easily determine the other key. One key is used to encrypt the plaintext and the other key is used to decrypt the ciphertext. The important point here is that it does not matter which key is applied first, but that both keys are required for the process to work (Figure 1B). Because a pair of keys are required, this approach is also called asymmetric cryptography.

In PKC, one of the keys is designated the public key and may be advertised as widely as the owner wants. The other key is designated the private key and is never revealed to another party. It is straight forward to send messages under this scheme. Suppose Alice wants to send Bob a message. Alice encrypts some information using Bob’s public key; Bob decrypts the ciphertext using his private key. This method could be also used to prove who sent a message; Alice, for example, could encrypt some plaintext with her private key; when Bob decrypts using Alice’s public key, he knows that Alice sent the message and Alice cannot deny having sent the message (non-repudiation).

Public-key cryptography algorithms that are in use today for key exchange or digital signatures include:

  • RSA: The first, and still most common, PKC implementation, named for the three MIT mathematicians who developed it — Ronald Rivest, Adi Shamir, and Leonard Adleman. RSA today is used in hundreds of software products and can be used for key exchange, digital signatures, or encryption of small blocks of data. RSA uses a variable size encryption block and a variable size key. The key-pair is derived from a very large number, n, that is the product of two prime numbers chosen according to special rules; these primes may be 100 or more digits in length each, yielding an n with roughly twice as many digits as the prime factors. The public key information includes n and a derivative of one of the factors of n; an attacker cannot determine the prime factors of n (and, therefore, the private key) from this information alone and that is what makes the RSA algorithm so secure. (Some descriptions of PKC erroneously state that RSA’s safety is due to the difficulty in factoring large prime numbers. In fact, large prime numbers, like small prime numbers, only have two factors!) The ability for computers to factor large numbers, and therefore attack schemes such as RSA, is rapidly improving and systems today can find the prime factors of numbers with more than 200 digits. Nevertheless, if a large number is created from two prime factors that are roughly the same size, there is no known factorization algorithm that will solve the problem in a reasonable amount of time; a 2005 test to factor a 200-digit number took 1.5 years and over 50 years of compute time (see the Wikipedia article on integer factorization.) Regardless, one presumed protection of RSA is that users can easily increase the key size to always stay ahead of the computer processing curve. As an aside, the patent for RSA expired in September 2000 which does not appear to have affected RSA’s popularity one way or the other. A detailed example of RSA is presented below in Section 5.3.
  • Diffie-Hellman: After the RSA algorithm was published, Diffie and Hellman came up with their own algorithm. D-H is used for secret-key key exchange only, and not for authentication or digital signatures. More detail about Diffie-Hellman can be found below in Section 5.2.
  • Digital Signature Algorithm (DSA): The algorithm specified in NIST’s Digital Signature Standard (DSS), provides digital signature capability for the authentication of messages.
  • ElGamal: Designed by Taher Elgamal, a PKC system similar to Diffie-Hellman and used for key exchange.
  • Elliptic Curve Cryptography (ECC): A PKC algorithm based upon elliptic curves. ECC can offer levels of security with small keys comparable to RSA and other PKC methods. It was designed for devices with limited compute power and/or memory, such as smartcards and PDAs. More detail about ECC can be found below in Section 5.8. Other references include “The Importance of ECC” Web page and the “Online Elliptic Curve Cryptography Tutorial”, both from Certicom. See also RFC 6090 for a review of fundamental ECC algorithms.
  • Public-Key Cryptography Standards (PKCS): A set of interoperable standards and guidelines for public-key cryptography, designed by RSA Data Security Inc.
    • PKCS #1: RSA Cryptography Standard (Also RFC 3447)
    • PKCS #2: Incorporated into PKCS #1.
    • PKCS #3: Diffie-Hellman Key-Agreement Standard
    • PKCS #4: Incorporated into PKCS #1.
    • PKCS #5: Password-Based Cryptography Standard (PKCS #5 V2.0 is also RFC 2898)
    • PKCS #6: Extended-Certificate Syntax Standard (being phased out in favor of X.509v3)
    • PKCS #7: Cryptographic Message Syntax Standard (Also RFC 2315)
    • PKCS #8: Private-Key Information Syntax Standard (Also RFC 5208)
    • PKCS #9: Selected Attribute Types (Also RFC 2985)
    • PKCS #10: Certification Request Syntax Standard (Also RFC 2986)
    • PKCS #11: Cryptographic Token Interface Standard
    • PKCS #12: Personal Information Exchange Syntax Standard
    • PKCS #13: Elliptic Curve Cryptography Standard
    • PKCS #14: Pseudorandom Number Generation Standard is no longer available
    • PKCS #15: Cryptographic Token Information Format Standard
  • Cramer-Shoup: A public-key cryptosystem proposed by R. Cramer and V. Shoup of IBM in 1998.
  • Key Exchange Algorithm (KEA): A variation on Diffie-Hellman; proposed as the key exchange method for Capstone.
  • LUC: A public-key cryptosystem designed by P.J. Smith and based on Lucas sequences. Can be used for encryption and signatures, using integer factoring.

For additional information on PKC algorithms, see “Public-Key Encryption”, Chapter 8 in Handbook of Applied Cryptography, by A. Menezes, P. van Oorschot, and S. Vanstone (CRC Press, 1996).


A digression: Who invented PKC? I tried to be careful in the first paragraph of this section to state that Diffie and Hellman “first described publicly” a PKC scheme. Although I have categorized PKC as a two-key system, that has been merely for convenience; the real criteria for a PKC scheme is that it allows two parties to exchange a secret even though the communication with the shared secret might be overheard. There seems to be no question that Diffie and Hellman were first to publish; their method is described in the classic paper, “New Directions in Cryptography,” published in the November 1976 issue of IEEE Transactions on Information Theory. As shown below, Diffie-Hellman uses the idea that finding logarithms is relatively harder than exponentiation. And, indeed, it is the precursor to modern PKC which does employ two keys. Rivest, Shamir, and Adleman described an implementation that extended this idea in their paper “A Method for Obtaining Digital Signatures and Public-Key Cryptosystems,” published in the February 1978 issue of the Communications of the ACM (CACM). Their method, of course, is based upon the relative ease of finding the product of two large prime numbers compared to finding the prime factors of a large number.

Some sources, though, credit Ralph Merkle with first describing a system that allows two parties to share a secret although it was not a two-key system, per se. A Merkle Puzzle works where Alice creates a large number of encrypted keys, sends them all to Bob so that Bob chooses one at random and then lets Alice know which he has selected. An eavesdropper will see all of the keys but can’t learn which key Bob has selected (because he has encrypted the response with the chosen key). In this case, Eve’s effort to break in is the square of the effort of Bob to choose a key. While this difference may be small it is often sufficient. Merkle apparently took a computer science course at UC Berkeley in 1974 and described his method, but had difficulty making people understand it; frustrated, he dropped the course. Meanwhile, he submitted the paper “Secure Communication Over Insecure Channels” which was published in the CACM in April 1978; Rivest et al.’s paper even makes reference to it. Merkle’s method certainly wasn’t published first, but did he have the idea first?

An interesting question, maybe, but who really knows? For some time, it was a quiet secret that a team at the UK’s Government Communications Headquarters (GCHQ) had first developed PKC in the early 1970s. Because of the nature of the work, GCHQ kept the original memos classified. In 1997, however, the GCHQ changed their posture when they realized that there was nothing to gain by continued silence. Documents show that a GCHQ mathematician named James Ellis started research into the key distribution problem in 1969 and that by 1975, Ellis, Clifford Cocks, and Malcolm Williamson had worked out all of the fundamental details of PKC, yet couldn’t talk about their work. (They were, of course, barred from challenging the RSA patent!) After more than 20 years, Ellis, Cocks, and Williamson have begun to get their due credit.

And the National Security Agency (NSA) claims to have knowledge of this type of algorithm as early as 1966 but there is no supporting documentation… yet. So this really was a digression…


3.3. Hash Functions

Hash functions, also called message digests and one-way encryption, are algorithms that, in some sense, use no key (Figure 1C). Instead, a fixed-length hash value is computed based upon the plaintext that makes it impossible for either the contents or length of the plaintext to be recovered. Hash algorithms are typically used to provide a digital fingerprint of a file’s contents, often used to ensure that the file has not been altered by an intruder or virus. Hash functions are also commonly employed by many operating systems to encrypt passwords. Hash functions, then, provide a measure of the integrity of a file.

Hash algorithms that are in common use today include:

  • Message Digest (MD) algorithms: A series of byte-oriented algorithms that produce a 128-bit hash value from an arbitrary-length message.
    • MD2 (RFC 1319): Designed for systems with limited memory, such as smart cards. (MD2 has been relegated to historical status, per RFC 6149.)
    • MD4 (RFC 1320): Developed by Rivest, similar to MD2 but designed specifically for fast processing in software. (MD4 has been relegated to historical status, per RFC 6150.)
    • MD5 (RFC 1321): Also developed by Rivest after potential weaknesses were reported in MD4; this scheme is similar to MD4 but is slower because more manipulation is made to the original data. MD5 has been implemented in a large number of products although several weaknesses in the algorithm were demonstrated by German cryptographer Hans Dobbertin in 1996 (“Cryptanalysis of MD5 Compress”).
  • Secure Hash Algorithm (SHA): Algorithm for NIST’s Secure Hash Standard (SHS). SHA-1 produces a 160-bit hash value and was originally published as FIPS 180-1 and RFC 3174FIPS 180-2(aka SHA-2) describes five algorithms in the SHS: SHA-1 plus SHA-224, SHA-256, SHA-384, and SHA-512 which can produce hash values that are 224, 256, 384, or 512 bits in length, respectively. SHA-224, -256, -384, and -512 are also described in RFC 4634.
  • RIPEMD: A series of message digests that initially came from the RIPE (RACE Integrity Primitives Evaluation) project. RIPEMD-160 was designed by Hans Dobbertin, Antoon Bosselaers, and Bart Preneel, and optimized for 32-bit processors to replace the then-current 128-bit hash functions. Other versions include RIPEMD-256, RIPEMD-320, and RIPEMD-128.
  • HAVAL (HAsh of VAriable Length): Designed by Y. Zheng, J. Pieprzyk and J. Seberry, a hash algorithm with many levels of security. HAVAL can create hash values that are 128, 160, 192, 224, or 256 bits in length.
  • Whirlpool: A relatively new hash function, designed by V. Rijmen and P.S.L.M. Barreto. Whirlpool operates on messages less than 2256 bits in length, and produces a message digest of 512 bits. The design of this has function is very different than that of MD5 and SHA-1, making it immune to the same attacks as on those hashes (see below).
  • Tiger: Designed by Ross Anderson and Eli Biham, Tiger is designed to be secure, run efficiently on 64-bit processors, and easily replace MD4, MD5, SHA and SHA-1 in other applications. Tiger/192 produces a 192-bit output and is compatible with 64-bit architectures; Tiger/128 and Tiger/160 produce a hash of length 128 and 160 bits, respectively, to provide compatibility with the other hash functions mentioned above.

(Readers might be interested in HashCalc, a Windows-based program that calculates hash values using a dozen algorithms, including MD5, SHA-1 and several variants, RIPEMD-160, and Tiger. Command line utilities that calculate hash values include sha_verify by Dan Mares [Windows; supports MD5, SHA-1, SHA-2] and md5deep [cross-platform; supports MD5, SHA-1, SHA-256, Tiger, and Whirlpool].)

Hash functions are sometimes misunderstood and some sources claim that no two files can have the same hash value. This is, in fact, not correct. Consider a hash function that provides a 128-bit hash value. There are, obviously, 2128 possible hash values. But there are a lot more than 2128 possible files. Therefore, there have to be multiple files — in fact, there have to be an infinite number of files! — that can have the same 128-bit hash value.

The difficulty is finding two files with the same hash! What is, indeed, very hard to do is to try to create a file that has a given hash value so as to force a hash value collision — which is the reason that hash functions are used extensively for information security and computer forensics applications. Alas, researchers in 2004 found that practical collision attacks could be launched on MD5, SHA-1, and other hash algorithms. Readers interested in this problem should read the following:

Readers are also referred to the Eindhoven University of Technology HashClash Project Web site. An excellent overview of the situation with hash collisions (circa 2005) can be found in RFC 4270 (by P. Hoffman and B. Schneier, November 2005). And for additional information on hash functions, see David Hopwood’s MessageDigest Algorithms page.

At this time, there is no obvious successor to MD5 and SHA-1 that could be put into use quickly; there are so many products using these hash functions that it could take many years to flush out all use of 128- and 160-bit hashes. That said, NIST announced in 2007 their Cryptographic Hash Algorithm Competition to find the next-generation secure hashing method. Dubbed SHA-3, this new scheme will augment FIPS 180-2. A list of submissions can be found at The SHA-3 Zoo. The SHA-3 standard may not be available until 2011 or 2012.

Certain extensions of hash functions are used for a variety of information security and digital forensics applications, such as:

  • Hash libraries are sets of hash values corresponding to known files. A hash library of known good files, for example, might be a set of files known to be a part of an operating system, while a hash library of known bad files might be of a set of known child pornographic images.
  • Rolling hashes refer to a set of hash values that are computed based upon a fixed-length “sliding window” through the input. As an example, a hash value might be computed on bytes 1-10 of a file, then on bytes 2-11, 3-12, 4-13, etc.
  • Fuzzy hashes are an area of intense research and represent hash values that represent two inputs that are similar. Fuzzy hashes are used to detect documents, images, or other files that are close to each other with respect to content. See “Fuzzy Hashing” (PDF | PPT) by Jesse Kornblum for a good treatment of this topic.

3.4. Why Three Encryption Techniques?

So, why are there so many different types of cryptographic schemes? Why can’t we do everything we need with just one?

The answer is that each scheme is optimized for some specific application(s). Hash functions, for example, are well-suited for ensuring data integrity because any change made to the contents of a message will result in the receiver calculating a different hash value than the one placed in the transmission by the sender. Since it is highly unlikely that two different messages will yield the same hash value, data integrity is ensured to a high degree of confidence.

Secret key cryptography, on the other hand, is ideally suited to encrypting messages, thus providing privacy and confidentiality. The sender can generate a session key on a per-message basis to encrypt the message; the receiver, of course, needs the same session key to decrypt the message.

Key exchange, of course, is a key application of public-key cryptography (no pun intended). Asymmetric schemes can also be used for non-repudiation and user authentication; if the receiver can obtain the session key encrypted with the sender’s private key, then only this sender could have sent the message. Public-key cryptography could, theoretically, also be used to encrypt messages although this is rarely done because secret-key cryptography operates about 1000 times faster than public-key cryptography.

FIGURE 2: Sample application of the three cryptographic techniques for secure communication.

Figure 2 puts all of this together and shows how a hybrid cryptographic scheme combines all of these functions to form a secure transmission comprising digital signature and digital envelope. In this example, the sender of the message is Alice and the receiver is Bob.

A digital envelope comprises an encrypted message and an encrypted session key. Alice uses secret key cryptography to encrypt her message using the session key, which she generates at random with each session. Alice then encrypts the session key using Bob’s public key. The encrypted message and encrypted session key together form the digital envelope. Upon receipt, Bob recovers the session secret key using his private key and then decrypts the encrypted message.

The digital signature is formed in two steps. First, Alice computes the hash value of her message; next, she encrypts the hash value with her private key. Upon receipt of the digital signature, Bob recovers the hash value calculated by Alice by decrypting the digital signature with Alice’s public key. Bob can then apply the hash function to Alice’s original message, which he has already decrypted (see previous paragraph). If the resultant hash value is not the same as the value supplied by Alice, then Bob knows that the message has been altered; if the hash values are the same, Bob should believe that the message he received is identical to the one that Alice sent.

This scheme also provides nonrepudiation since it proves that Alice sent the message; if the hash value recovered by Bob using Alice’s public key proves that the message has not been altered, then only Alice could have created the digital signature. Bob also has proof that he is the intended receiver; if he can correctly decrypt the message, then he must have correctly decrypted the session key meaning that his is the correct private key.

3.5. The Significance of Key Length

In a recent article in the industry literature (circa 9/98), a writer made the claim that 56-bit keys do not provide as sufficient protection for DES today as they did in 1975 because computers are 1000 times faster today than in 1975. Therefore, the writer went on, we should be using 56,000-bit keys today instead of 56-bit keys to provide adequate protection. The conclusion was then drawn that because 56,000-bit keys are infeasible (true), we should accept the fact that we have to live with weak cryptography (false!). The major error here is that the writer did not take into account that the number of possible key values double whenever a single bit is added to the key length; thus, a 57-bit key has twice as many values as a 56-bit key (because 257 is two times 256). In fact, a 66-bit key would have 1024 times the possible values as a 56-bit key.

But this does bring up the issue, what is the precise significance of key length as it affects the level of protection?

In cryptography, size does matter. The larger the key, the harder it is to crack a block of encrypted data. The reason that large keys offer more protection is almost obvious; computers have made it easier to attack ciphertext by using brute force methods rather than by attacking the mathematics (which are generally well-known anyway). With a brute force attack, the attacker merely generates every possible key and applies it to the ciphertext. Any resulting plaintext that makes sense offers a candidate for a legitimate key. This was the basis, of course, of the EFF’s attack on DES.

Until the mid-1990s or so, brute force attacks were beyond the capabilities of computers that were within the budget of the attacker community. Today, however, significant compute power is commonly available and accessible. General purpose computers such as PCs are already being used for brute force attacks. For serious attackers with money to spend, such as some large companies or governments, Field Programmable Gate Array (FPGA) or Application-Specific Integrated Circuits (ASIC) technology offers the ability to build specialized chips that can provide even faster and cheaper solutions than a PC. Consider that an AT&T ORCA chip (FPGA) costs $200 and can test 30 million DES keys per second, while a $10 ASIC chip can test 200 million DES keys per second (compared to a PC which might be able to test 40,000 keys per second).

The table below shows what DES key sizes are needed to protect data from attackers with different time and financial resources. This information is not merely academic; one of the basic tenets of any security system is to have an idea of what you are protecting and from who are you protecting it! The table clearly shows that a 40-bit key is essentially worthless today against even the most unsophisticated attacker. On the other hand, 56-bit keys are fairly strong unless you might be subject to some pretty serious corporate or government espionage. But note that even 56-bit keys are declining in their value and that the times in the table (1995 data) are worst cases.

TABLE 1. Minimum Key Lengths for Symmetric Ciphers.
Type of Attacker Budget Tool Time and Cost
Per Key Recovered
Key Length Needed
For Protection
In Late-1995
40 bits 56 bits
Pedestrian Hacker Tiny Scavanged
computer
time
1 week Infeasible 45
$400 FPGA 5 hours
($0.08)
38 years
($5,000)
50
Small Business $10,000 FPGA 12 minutes
($0.08)
18 months
($5,000)
55
Corporate Department $300K FPGA 24 seconds
($0.08)
19 days
($5,000)
60
ASIC 0.18 seconds
($0.001)
3 hours
($38)
Big Company $10M FPGA 7 seconds
($0.08)
13 hours
($5,000)
70
ASIC 0.005 seconds
($0.001)
6 minutes
($38)
Intelligence Agency $300M ASIC 0.0002 seconds
($0.001)
12 seconds
($38)
75

So, how big is big enough? DES, invented in 1975, is still in use today, nearly 25 years later. If we take that to be a design criteria (i.e., a 20-plus year lifetime) and we believe Moore’s Law (“computing power doubles every 18 months”), then a key size extension of 14 bits (i.e., a factor of more than 16,000) should be adequate. The 1975 DES proposal suggested 56-bit keys; by 1995, a 70-bit key would have been required to offer equal protection and an 85-bit key will be necessary by 2015.

The discussion above suggests that a 128- or 256-bit key for SKC will suffice for some time because that key length keeps us ahead of the brute force capabilities of the attackers. While a large key is good, a huge key may not always be better. That is, many public-key cryptosystems use 1024- or 2048-bit keys; expanding the key to 4096 bits probably doesn’t add any protection at this time but it does add significantly to processing time.

The most effective large-number factoring methods today use a mathematical Number Field Sieve to find a certain number of relationships and then uses a matrix operation to solve a linear equation to produce the two prime factors. The sieve step actually involves a large number of operations of operations that can be performed in parallel; solving the linear equation, however, requires a supercomputer. Indeed, finding the solution to the RSA-140 challenge in February 1999 — factoring a 140-digit (465-bit) prime number — required 200 computers across the Internet about 4 weeks for the first step and a Cray computer 100 hours and 810 MB of memory to do the second step.

In early 1999, Shamir (of RSA fame) described a new machine that could increase factorization speed by 2-3 orders of magnitude. Although no detailed plans were provided nor is one known to have been built, the concepts of TWINKLE (The Weizmann Institute Key Locating Engine) could result in a specialized piece of hardware that would cost about $5000 and have the processing power of 100-1000 PCs. There still appear to be many engineering details that have to be worked out before such a machine could be built. Furthermore, the hardware improves the sieve step only; the matrix operation is not optimized at all by this design and the complexity of this step grows rapidly with key length, both in terms of processing time and memory requirements. Nevertheless, this plan conceptually puts 512-bit keys within reach of being factored. Although most PKC schemes allow keys that are 1024 bits and longer, Shamir claims that 512-bit RSA keys “protect 95% of today’s E-commerce on the Internet.” (See Bruce Schneier’sCrypto-Gram (May 15, 1999) for more information, as well as the comments from RSA Labs.)

It is also interesting to note that while cryptography is good and strong cryptography is better, long keys may disrupt the nature of the randomness of data files. Shamir and van Someren (“Playing hide and seek with stored keys”) have noted that a new generation of viruses can be written that will find files encrypted with long keys, making them easier to find by intruders and, therefore, more prone to attack.

Finally, U.S. government policy has tightly controlled the export of crypto products since World War II. Until recently, export outside of North America of cryptographic products using keys greater than 40 bits in length was prohibited, which made those products essentially worthless in the marketplace, particularly for electronic commerce. More recently, the U.S. Commerce Department relaxed the regulations, allowing the general export of 56-bit SKC and 1024-bit PKC products (certain sectors, such as health care and financial, allow the export of products with even larger keys). The Commerce Department’s Bureau of Export Administration maintains a Commercial Encryption Export Controls web page with more information. The potential impact of this policy on U.S. businesses is well beyond the scope of this paper.

Much of the discussion above, including the table, are based on the paper “Minimal Key Lengths for Symmetric Ciphers to Provide Adequate Commercial Security” by M. Blaze, W. Diffie, R.L. Rivest, B. Schneier, T. Shimomura, E. Thompson, and M. Wiener.

On a related topic, public key crypto schemes can be used for several purposes, including key exchange, digital signatures, authentication, and more. In those PKC systems used for SKC key exchange, the PKC key lengths are chosen so to be resistant to some selected level of attack. The length of the secret keys exchanged via that system have to have at least the same level of attack resistance. Thus, the three parameters of such a system — system strength, secret key strength, and public key strength — must be matched. This topic is explored in more detail in Determining Strengths For Public Keys Used For Exchanging Symmetric Keys (RFC 3766).

4. TRUST MODELS

Secure use of cryptography requires trust. While secret key cryptography can ensure message confidentiality and hash codes can ensure integrity, none of this works without trust. In SKC, Alice and Bob had to share a secret key. PKC solved the secret distribution problem, but how does Alice really know that Bob is who he says he is? Just because Bob has a public and private key, and purports to be “Bob,” how does Alice know that a malicious person (Mallory) is not pretending to be Bob?

There are a number of trust models employed by various cryptographic schemes. This section will explore three of them:

  • The web of trust employed by Pretty Good Privacy (PGP) users, who hold their own set of trusted public keys.
  • Kerberos, a secret key distribution scheme using a trusted third party.
  • Certificates, which allow a set of trusted third parties to authenticate each other and, by implication, each other’s users.

Each of these trust models differs in complexity, general applicability, scope, and scalability.

4.1. PGP Web of Trust

Pretty Good Privacy (described more below in Section 5.5) is a widely used private e-mail scheme based on public key methods. A PGP user maintains a local keyring of all their known and trusted public keys. The user makes their own determination about the trustworthiness of a key using what is called a “web of trust.”

If Alice needs Bob’s public key, Alice can ask Bob for it in another e-mail or, in many cases, download the public key from an advertised server; this server might a well-known PGP key repository or a site that Bob maintains himself. In fact, Bob’s public key might be stored or listed in many places. (The author’s public key, for example, can be found at http://www.garykessler.net/kumquat_pubkey.html.) Alice is prepared to believe that Bob’s public key, as stored at these locations, is valid.

Suppose Carol claims to hold Bob’s public key and offers to give the key to Alice. How does Alice know that Carol’s version of Bob’s key is valid or if Carol is actually giving Alice a key that will allow Mallory access to messages? The answer is, “It depends.” If Alice trusts Carol and Carol says that she thinks that her version of Bob’s key is valid, then Alice may — at her option — trust that key. And trust is not necessarily transitive; if Dave has a copy of Bob’s key and Carol trusts Dave, it does not necessarily follow that Alice trusts Dave even if she does trust Carol.

The point here is that who Alice trusts and how she makes that determination is strictly up to Alice. PGP makes no statement and has no protocol about how one user determines whether they trust another user or not. In any case, encryption and signatures based on public keys can only be used when the appropriate public key is on the user’s keyring.

4.2. Kerberos

Kerberos is a commonly used authentication scheme on the Internet. Developed by MIT’s Project Athena, Kerberos is named for the three-headed dog who, according to Greek mythology, guards the entrance of Hades (rather than the exit, for some reason!).

Kerberos employs a client/server architecture and provides user-to-server authentication rather than host-to-host authentication. In this model, security and authentication will be based on secret key technology where every host on the network has its own secret key. It would clearly be unmanageable if every host had to know the keys of all other hosts so a secure, trusted host somewhere on the network, known as a Key Distribution Center (KDC), knows the keys for all of the hosts (or at least some of the hosts within a portion of the network, called a realm). In this way, when a new node is brought online, only the KDC and the new node need to be configured with the node’s key; keys can be distributed physically or by some other secure means.

FIGURE 3: Kerberos architecture.

The Kerberos Server/KDC has two main functions (Figure 3), known as the Authentication Server (AS) and Ticket-Granting Server (TGS). The steps in establishing an authenticated session between an application client and the application server are: 

  1. The Kerberos client software establishes a connection with the Kerberos server’s AS function. The AS first authenticates that the client is who it purports to be. The AS then provides the client with a secret key for this login session (the TGS session key) and a ticket-granting ticket (TGT), which gives the client permission to talk to the TGS. The ticket has a finite lifetime so that the authentication process is repeated periodically.
  2. The client now communicates with the TGS to obtain the Application Server’s key so that it (the client) can establish a connection to the service it wants. The client supplies the TGS with the TGS session key and TGT; the TGS responds with an application session key (ASK) and an encrypted form of the Application Server’s secret key; this secret key is never sent on the network in any other form.
  3. The client has now authenticated itself and can prove its identity to the Application Server by supplying the Kerberos ticket, application session key, and encrypted Application Server secret key. The Application Server responds with similarly encrypted information to authenticate itself to the client. At this point, the client can initiate the intended service requests (e.g., Telnet, FTP, HTTP, or e-commerce transaction session establishment).

The current shipping version of this protocol is Kerberos V5 (described in RFC 1510), although Kerberos V4 still exists and is seeing some use. While the details of their operation, functional capabilities, and message formats are different, the conceptual overview above pretty much holds for both. One primary difference is that Kerberos V4 uses only DES to generate keys and encrypt messages, while V5 allows other schemes to be employed (although DES is still the most widely algorithm used).

4.3. Public Key Certificates and Certificate Authorities

Certificates and Certificate Authorities (CA) are necessary for widespread use of cryptography for e-commerce applications. While a combination of secret and public key cryptography can solve the business issues discussed above, crypto cannot alone address the trust issues that must exist between a customer and vendor in the very fluid, very dynamic e-commerce relationship. How, for example, does one site obtain another party’s public key? How does a recipient determine if a public key really belongs to the sender? How does the recipient know that the sender is using their public key for a legitimate purpose for which they are authorized? When does a public key expire? How can a key be revoked in case of compromise or loss?

The basic concept of a certificate is one that is familiar to all of us. A driver’s license, credit card, or SCUBA certification, for example, identify us to others, indicate something that we are authorized to do, have an expiration date, and identify the authority that granted the certificate.

As complicated as this may sound, it really isn’t! Consider driver’s licenses. I have one issued by the State of Vermont. The license establishes my identity, indicates the type of vehicles that I can operate and the fact that I must wear corrective lenses while doing so, identifies the issuing authority, and notes that I am an organ donor. When I drive outside of Vermont, the other jurisdictions throughout the U.S. recognize the authority of Vermont to issue this “certificate” and they trust the information it contains. Now, when I leave the U.S., everything changes. When I am in Canada and many other countries, they will accept not the Vermont license, per se, but any license issued in the U.S.; some other countries may not recognize the Vermont driver’s license as sufficient bona fides that I can drive. This analogy represents the certificate chain, where even certificates carry certificates.

For purposes of electronic transactions, certificates are digital documents. The specific functions of the certificate include:

  • Establish identity: Associate, or bind, a public key to an individual, organization, corporate position, or other entity.
  • Assign authority: Establish what actions the holder may or may not take based upon this certificate.
  • Secure confidential information (e.g., encrypting the session’s symmetric key for data confidentiality).

Typically, a certificate contains a public key, a name, an expiration date, the name of the authority that issued the certificate (and, therefore, is vouching for the identity of the user), a serial number, any pertinent policies describing how the certificate was issued and/or how the certificate may be used, the digital signature of the certificate issuer, and perhaps other information.

FIGURE 4: GTE Cybertrust Global Root-issued certificate as viewed
by Netscape Navigator V4.

A sample abbreviated certificate is shown in Figure 4. This is a typical certificate found in a browser; while this one is issued by GTE Cybertrust, many so-called root-level certificates can be found shipped with browsers. When the browser makes a connection to a secure Web site, the Web server sends its public key certificate to the browser. The browser then checks the certificate’s signature against the public key that it has stored; if there is a match, the certificate is taken as valid and the Web site verified by this certificate is considered to be “trusted.”

TABLE 2. Contents of an X.509 V3 Certificate.
      version number
      certificate serial number
      signature algorithm identifier
      issuer’s name and unique identifier
      validity (or operational) period
      subject’s name and unique identifier
      subject public key information
      standard extensions
        certificate appropriate use definition
        key usage limitation definition
        certificate policy information
      other extensions
        Application-specific
        CA-specific

The most widely accepted certificate format is the one defined in International Telecommunication Union Telecommunication Standardization Sector (ITU-T) Recommendation X.509. Rec. X.509 is a specification used around the world and any applications complying with X.509 can share certificates. Most certificates today comply with X.509 Version 3 and contain the information listed in Table 2.

Certificate authorities are the repositories for public-keys and can be any agency that issues certificates. A company, for example, may issue certificates to its employees, a college/university to its students, a store to its customers, an Internet service provider to its users, or a government to its constituents.

When a sender needs an intended receiver’s public key, the sender must get that key from the receiver’s CA. That scheme is straight-forward if the sender and receiver have certificates issued by the same CA. If not, how does the sender know to trust the foreign CA? One industry wag has noted, about trust: “You are either born with it or have it granted upon you.” Thus, some CAs will be trusted because they are known to be reputable, such as the CAs operated by AT&T, BBN, Canada Post Corp., CommerceNet, GTE Cybertrust, MCI, Nortel EnTrustThawte, the U.S. Postal Service, and VeriSign. CAs, in turn, form trust relationships with other CAs. Thus, if a user queries a foreign CA for information, the user may ask to see a list of CAs that establish a “chain of trust” back to the user.

One major feature to look for in a CA is their identification policies and procedures. When a user generates a key pair and forwards the public key to a CA, the CA has to check the sender’s identification and takes any steps necessary to assure itself that the request is really coming from the advertised sender. Different CAs have different identification policies and will, therefore, be trusted differently by other CAs. Verification of identity is just of many issues that are part of a CA’s Certification Practice Statement (CPS) and policies; other issues include how the CA protects the public keys in its care, how lost or compromised keys are revoked, and how the CA protects its own private keys.

4.4. Summary

The paragraphs above describe three very different trust models. It is hard to say that any one is better than the others; it depend upon your application. One of the biggest and fastest growing applications of cryptography today, though, is electronic commerce (e-commerce), a term that itself begs for a formal definition.

PGP’s web of trust is easy to maintain and very much based on the reality of users as people. The model, however, is limited; just how many public keys can a single user reliably store and maintain? And what if you are using the “wrong” computer when you want to send a message and can’t access your keyring? How easy it is to revoke a key if it is compromised? PGP may also not scale well to an e-commerce scenario of secure communication between total strangers on short-notice.

Kerberos overcomes many of the problems of PGP’s web of trust, in that it is scalable and its scope can be very large. However, it also requires that the Kerberos server have a priori knowledge of all client systems prior to any transactions, which makes it unfeasible for “hit-and-run” client/server relationships as seen in e-commerce.

Certificates and the collection of CAs will form a Public Key Infrastructure (PKI). In the early days of the Internet, every host had to maintain a list of every other host; the Domain Name System (DNS) introduced the idea of a distributed database for this purpose and the DNS is one of the key reasons that the Internet has grown as it has. A PKI will fill a similar void in the e-commerce and PKC realm.

While certificates and the benefits of a PKI are most often associated with electronic commerce, the applications for PKI are much broader and include secure electronic mail, payments and electronic checks, Electronic Data Interchange (EDI), secure transfer of Domain Name System (DNS) and routing information, electronic forms, and digitally signed documents. A single “global PKI” is still many years away, that is the ultimate goal of today’s work as international electronic commerce changes the way in which we do business in a similar way in which the Internet has changed the way in which we communicate.

5. CRYPTOGRAPHIC ALGORITHMS IN ACTION

The paragraphs above have provided an overview of the different types of cryptographic algorithms, as well as some examples of some available protocols and schemes. Table 3 provides a list of some other noteworthy schemes employed — or proposed — for a variety of functions, most notably electronic commerce. The paragraphs below will show several real cryptographic applications that many of us employ (knowingly or not) everyday for password protection and private communication.

TABLE 3. Other Crypto Algorithms and Systems of Note.

Capstone A now-defunct U.S. National Institute of Standards and Technology (NIST) and National Security Agency (NSA) project under the Bush Sr. and Clinton administrations for publicly available strong cryptography with keys escrowed by the government (NIST and the Treasury Dept.). Capstone included in one or more tamper-proof computer chips for implementation (Clipper), a secret key encryption algorithm (Skipjack), digital signature algorithm (DSA), key exchange algorithm (KEA), and hash algorithm (SHA).
Clipper The computer chip that would implement the Skipjack encryption scheme. See also EPIC’s The Clipper Chip Web page.
Escrowed Encryption Standard (EES) Largely unused, a controversial crypto scheme employing the SKIPJACK secret key crypto algorithm and a Law Enforcement Access Field (LEAF) creation method. LEAF was one part of the key escrow system and allowed for decryption of ciphertext messages that had been legally intercepted by law enforcement agencies. Described more in FIPS 185.
Federal Information Processing Standards (FIPS) These computer security- and crypto-related FIPS are produced by the U.S. National Institute of Standards and Technology (NIST) as standards for the U.S. Government.
Fortezza (formerly called Tessera) A PCMCIA card developed by NSA that implements the Capstone algorithms, intended for use with the Defense Messaging Service (DMS).
GOST GOST is a family of algorithms that is defined in the Russian cryptographic standards. Although most of the specifications are written in Russian, a series of RFCs describe some of the aspects so that the algorithms can be used effectively in Internet applications: 

  • RFC 4357: Additional Cryptographic Algorithms for Use with GOST 28147-89, GOST R 34.10-94, GOST R 34.10-2001, and GOST R 34.11-94 Algorithms
  • RFC 5830: GOST 28147-89: Encryption, Decryption, and Message Authentication Code (MAC) Algorithms
  • RFC 5831: GOST R 34.11-94: Hash Function Algorithm
  • RFC 5832: GOST R 34.10-2001: Digital Signature Algorithm
Identity-Based Encryption (IBE) Identity-Based Encryption was first proposed by Adi Shamir in 1984, and is a key authentication system where the public key can be derived from some unique information based upon the user’s identity. In 2001, Dan Boneh (Stanford) and Matt Franklin (U.C., Davis) developed a practical implementation of IBE based on elliptic curves and a mathematical construct called the Weil Pairing. In that year, Clifford Cocks (GCHQ) also described another IBE solution based on quadratic residues in composite groups.

IP Security Protocol (IPsec) The IPsec protocol suite is used to provide privacy and authentication services at the IP layer. An overview of the protocol suite and of the documents comprising IPsec can be found in RFC 2411. Other documents include:

  • RFC 4301: IP security architecture.
  • RFC 4302: IP Authentication Header (AH), one of the two primary IPsec functions; AH provides connectionless integrity and data origin authentication for IP datagrams and protects against replay attacks.
  • RFC 4303: IP Encapsulating Security Payload (ESP), the other primary IPsec function; ESP provides a variety of security services within IPsec.
  • RFC 4304: Extended Sequence Number (ESN) Addendum, allows for negotiation of a 32- or 64- bit sequence number, used to detect replay attacks.
  • RFC 4305: Cryptographic algorithm implementation requirements for ESP and AH.
  • RFC 5996: The Internet Key Exchange (IKE) protocol, version 2, providing for mutual authentication and establishing and maintaining security associations.
    • IKE v1 was described in three separate documents, RFC 2407 (application of ISAKMP to IPsec), RFC 2408 (ISAKMP, a framework for key management and security associations), and RFC 2409 (IKE, using part of Oakley and part of SKEME in conjunction with ISAKMP to obtain authenticated keying material for use with ISAKMP, and for other security associations such as AH and ESP). IKE v1 is obsoleted with the introdcution of IKEv2.
  • RFC 4307: Cryptographic algoritms used with IKEv2.
  • RFC 4308: Crypto suites for IPsec, IKE, and IKEv2.
  • RFC 4309: The use of AES in CBC-MAC mode with IPsec ESP.
  • RFC 4312: The use of the Camellia cipher algorithm in IPsec.
  • RFC 4359: The Use of RSA/SHA-1 Signatures within Encapsulating Security Payload (ESP) and Authentication Header (AH).
  • RFC 4434: Describes AES-XCBC-PRF-128, a pseudo-random function derived from the AES for use with IKE.
  • RFC 2403: Describes use of the HMAC with MD5 algorithm for data origin authentication and integrity protection in both AH and ESP.
  • RFC 2405: Describes use of DES-CBC (DES in Cipher Block Chaining Mode) for confidentiality in ESP.
  • RFC 2410: Defines use of the NULL encryption algorithm (i.e., provides authentication and integrity without confidentiality) in ESP.
  • RFC 2412: Describes OAKLEY, a key determination and distribution protocol.
  • RFC 2451: Describes use of Cipher Block Chaining (CBC) mode cipher algorithms with ESP.
  • RFCs 2522 and 2523: Description of Photuris, a session-key management protocol for IPsec.

IPsec was first proposed for use with IP version 6 (IPv6), but can also be employed with the current IP version, IPv4.

(See more detail about IPsec below in Section 5.6.)

Internet Security Association and Key Management Protocol (ISAKMP/OAKLEY) ISAKMP/OAKLEY provide an infrastructure for Internet secure communications. ISAKMP, designed by the National Security Agency (NSA) and described in RFC 2408, is a framework for key management and security associations, independent of the key generation and cryptographic algorithms actually employed. The OAKLEY Key Determination Protocol, described in RFC 2412, is a key determination and distribution protocol using a variation of Diffie-Hellman.
Kerberos A secret-key encryption and authentication system, designed to authenticate requests for network resources within a user domain rather than to authenticate messages. Kerberos also uses a trusted third-party approach; a client communications with the Kerberos server to obtain “credentials” so that it may access services at the application server. Kerberos V4 uses DES to generate keys and encrypt messages; DES is also commonly used in Kerberos V5, although other schemes could be employed.Microsoft added support for Kerberos V5 — with some proprietary extensions — in Windows 2000. There are many Kerberos articles posted at Microsoft’s Knowledge Base, notably “Basic Overview of Kerberos User Authentication Protocol in Windows 2000,” “Windows 2000 Kerberos 5 Ticket Flags and KDC Options for AS_REQ and TGS_REQ Messages,” and “Kerberos Administration in Windows 2000.”
Keyed-Hash Message Authentication Code (HMAC) A message authentication scheme based upon secret key cryptography and the secret key shared between two parties rather than public key methods. Described in FIPS 198 and RFC 2104.
Message Digest Cipher (MDC) Invented by Peter Gutman, MDC turns a one-way hash function into a block cipher.
MIME Object Security Standard (MOSS) Designed as a successor to PEM to provide PEM-based security services to MIME messages.
NSA Suite B Cryptography An NSA standard for securing information at the SECRET level. Defines use of:

  • Advanced Encryption Standard (AES) with key sizes of 128 and 256 bits, per FIPS PUB 197 for encryption
  • The Ephemeral Unified Model and the One-Pass Diffie Hellman (referred to as ECDH) using the curves with 256 and 384-bit prime moduli, per NIST Special Publication 800-56A for key exchange
  • Elliptic Curve Digital Signature Algorithm (ECDSA) using the curves with 256 and 384-bit prime moduli, per FIPS PUB 186-3 for digital signatures
  • Secure Hash Algorithm (SHA) using 256 and 384 bits, per FIPS PUB 180-3 for hashing

RFC 6239 describes Suite B Cryptographic Suites for Secure Shell (SSH).

Pretty Good Privacy (PGP) A family of cryptographic routines for e-mail and file storage applications developed by Philip Zimmermann. PGP 2.6.x uses RSA for key management and digital signatures, IDEA for message encryption, and MD5 for computing the message’s hash value; more information can also be found in RFC 1991. PGP 5.x (formerly known as “PGP 3”) uses Diffie-Hellman/DSS for key management and digital signatures; IDEA, CAST, or 3DES for message encryption; and MD5 or SHA for computing the message’s hash value. OpenPGP, described in RFC 2440, is an open definition of security software based on PGP 5.x.(See more detail about PGP below in Section 5.5.)
Privacy Enhanced Mail (PEM) Provides secure electronic mail over the Internet and includes provisions for encryption (DES), authentication, and key management (DES, RSA). May be superseded by S/MIME and PEM-MIME. Developed by IETF PEM Working Group and defined in four RFCs:

  • RFC 1421: Part I, Message Encryption and Authentication Procedures
  • RFC 1422: Part II, Certificate-Based Key Management
  • RFC 1423: Part III, Algorithms, Modes, and Identifiers
  • RFC 1424: Part IV, Key Certification and Related Services
Private Communication Technology (PCT) Developed by Microsoft and Visa for secure communication on the Internet. Similar to SSL, PCT supports Diffie-Hellman, Fortezza, and RSA for key establishment; DES, RC2, RC4, and triple-DES for encryption; and DSA and RSA message signatures. A companion to SET.
Secure Electronic Transactions (SET) A merging of two other protocols: SEPP (Secure Electronic Payment Protocol), an open specification for secure bank card transactions over the Internet, developed by CyberCash, GTE, IBM, MasterCard, and Netscape; and STT (Secure Transaction Technology), a secure payment protocol developed by Microsoft and Visa International. Supports DES and RC4 for encryption, and RSA for signatures, key exchange, and public-key encryption of bank card numbers. SET is a companion to the PCT protocol.
Secure Hypertext Transfer Protocol (S-HTTP) An extension to HTTP to provide secure exchange of documents over the World Wide Web. Supported algorithms include RSA and Kerberos for key exchange, DES, IDEA, RC2, and Triple-DES for encryption.
Secure Multipurpose Internet Mail Extensions (S/MIME) An IETF secure e-mail scheme intended to supercede PEM. S/MIME, described in RFCs 2311 and 2312, adds digital signature and encryption capability to Internet MIME messages.
Secure Sockets Layer (SSL) Developed by Netscape Communications to provide application-independent security and privacy over the Internet. SSL is designed so that protocols such as HTTP, FTP (File Transfer Protocol), and Telnet can operate over it transparently. SSL allows both server authentication (mandatory) and client authentication (optional). RSA is used during negotiation to exchange keys and identify the actual cryptographic algorithm (DES, IDEA, RC2, RC4, or 3DES) to use for the session. SSL also uses MD5 for message digests and X.509 public-key certificates. (Found to be breakable soon after the IETF announced formation of group to work on TLS.)(See more detail about SSL below in Section 5.7.)
Server Gated Cryptography (SGC) Microsoft extension to SSL that provides strong encryption for online banking and other financial applications using RC2 (128-bit key), RC4 (128-bit key), DES (56-bit key), or 3DES (equivalent of 168-bit key). Use of SGC requires a Windows NT Server running Internet Information Server (IIS) 4.0 with a valid SGC certificate. SGC is available in 32-bit Windows versions of Internet Explorer (IE) 4.0, and support for Mac, Unix, and 16-bit Windows versions of IE is expected in the future.
Simple Authentication and Security Layer (SASL) (SASL) is a framework for providing authentication and data security services in connection-oriented protocols (a la TCP). It provides a structured interface and allows new protocols to reuse existing authentication mechanisms and allows old protocols to make use of new mechanisms.It has been common practice on the Internet to permit anonymous access to various services, employing a plain-text password using a user name of “anonymous” and a password of an email address or some other identifying information. New IETF protocols disallow plain-text logins. The Anonymous SASL Mechanism (RFC 4505) provides a method for anonymous logins within the SASL framework.
Simple Key-Management for Internet Protocol (SKIP) Key management scheme for secure IP communication, specifically for IPsec, and designed by Aziz and Diffie. SKIP essentially defines a public key infrastructure for the Internet and even uses X.509 certificates. Most public key cryptosystems assign keys on a per-session basis, which is inconvenient for the Internet since IP is connectionless. Instead, SKIP provides a basis for secure communication between any pair of Internet hosts. SKIP can employ DES, 3DES, IDEA, RC2, RC5, MD5, and SHA-1.
Transport Layer Security (TLS) IETF specification (RFC 2246) intended to replace SSL. Employs Triple-DES (secret key cryptography), SHA (hash), Diffie-Hellman (key exchange), and DSS (digital signatures).(See more detail about TLS below in Section 5.7.)
TrueCrypt Open source, multi-platform cryptography software that can be used to encrypt a file, partition, or entire disk. One of TrueCrypt’s more interesting features is that of plausible deniability with hidden volumes or hidden operating systems.(See more detail about TrueCrypt below in Section 5.11.)
X.509 ITU-T recommendation for the format of certificates for the public key infrastructure. Certificates map (bind) a user identity to a public key. The IETF application of X.509 certificates is documented in RFC 2459. An Internet X.509 Public Key Infrastructure is further defined in RFC 2510 (Certificate Management Protocols) and RFC 2527 (Certificate Policy and Certification Practices Framework).

5.1. Password Protection

Nearly all modern multiuser computer and network operating systems employ passwords at the very least to protect and authenticate users accessing computer and/or network resources. But passwords arenot typically kept on a host or server in plaintext, but are generally encrypted using some sort of hash scheme.

A) /etc/passwd file

 root:Jbw6BwE4XoUHo:0:0:root:/root:/bin/bash
 carol:FM5ikbQt1K052:502:100:Carol Monaghan:/home/carol:/bin/bash
 alex:LqAi7Mdyg/HcQ:503:100:Alex Insley:/home/alex:/bin/bash
 gary:FkJXupRyFqY4s:501:100:Gary Kessler:/home/gary:/bin/bash
 todd:edGqQUAaGv7g6:506:101:Todd Pritsky:/home/todd:/bin/bash
 josh:FiH0ONcjPut1g:505:101:Joshua Kessler:/home/webroot:/bin/bash

B.1) /etc/passwd file (with shadow passwords)

 root:x:0:0:root:/root:/bin/bash
 carol:x:502:100:Carol Monaghan:/home/carol:/bin/bash
 alex:x:503:100:Alex Insley:/home/alex:/bin/bash
 gary:x:501:100:Gary Kessler:/home/gary:/bin/bash
 todd:x:506:101:Todd Pritsky:/home/todd:/bin/bash
 josh:x:505:101:Joshua Kessler:/home/webroot:/bin/bash

B.2) /etc/shadow file

 root:AGFw$1$P4u/uhLK$l2.HP35rlu65WlfCzq:11449:0:99999:7:::
 carol:kjHaN%35a8xMM8a/0kMl1?fwtLAM.K&kw.:11449:0:99999:7:::
 alex:1$1KKmfTy0a7#3.LL9a8H71lkwn/.hH22a:11449:0:99999:7:::
 gary:9ajlknknKJHjhnu7298ypnAIJKL$Jh.hnk:11449:0:99999:7:::
 todd:798POJ90uab6.k$klPqMt%alMlprWqu6$.:11492:0:99999:7:::
 josh:Awmqpsui*787pjnsnJJK%aappaMpQo07.8:11492:0:99999:7:::

 

FIGURE 5: Sample entries in Unix/Linux password files.

Unix/Linux, for example, uses a well-known hash via its crypt() function. Passwords are stored in the /etc/passwd file (Figure 5A); each record in the file contains the username, hashed password, user’s individual and group numbers, user’s name, home directory, and shell program; these fields are separated by colons (:). Note that each password is stored as a 13-byte string. The first two characters are actually a salt, randomness added to each password so that if two users have the same password, they will still be encrypted differently; the salt, in fact, provides a means so that a single password might have 4096 different encryptions. The remaining 11 bytes are the password hash, calculated using DES.

As it happens, the /etc/passwd file is world-readable on Unix systems. This fact, coupled with the weak encryption of the passwords, resulted in the development of the shadow password system where passwords are kept in a separate, non-world-readable file used in conjunction with the normal password file. When shadow passwords are used, the password entry in /etc/passwd is replaced with a “*” or “x” (Figure 5B.1) and the MD5 hash of the passwords are stored in /etc/shadow along with some other account information (Figure 5B.2).
Windows NT uses a similar scheme to store passwords in the Security Access Manager (SAM) file. In the NT case, all passwords are hashed using the MD4 algorithm, resulting in a 128-bit (16-byte) hash value (they are then obscured using an undocumented mathematical transformation that was a secret until distributed on the Internet). The password password, for example, might be stored as the hash value (in hexadecimal) 60771b22d73c34bd4a290a79c8b09f18.

Passwords are not saved in plaintext on computer systems precisely so they cannot be easily compromised. For similar reasons, we don’t want passwords sent in plaintext across a network. But for remote logon applications, how does a client system identify itself or a user to the server? One mechanism, of course, is to send the password as a hash value and that, indeed, may be done. A weakness of that approach, however, is that an intruder can grab the password off of the network and use an off-line attack (such as a dictionary attack where an attacker takes every known word and encrypts it with the network’s encryption algorithm, hoping eventually to find a match with a purloined password hash). In some situations, an attacker only has to copy the hashed password value and use it later on to gain unauthorized entry without ever learning the actual password.

An even stronger authentication method uses the password to modify a shared secret between the client and server, but never allows the password in any form to go across the network. This is the basis for the Challenge Handshake Authentication Protocol (CHAP), the remote logon process used by Windows NT.

As suggested above, Windows NT passwords are stored in a security file on a server as a 16-byte hash value. In truth, Windows NT stores two hashes; a weak hash based upon the old LAN Manager (LanMan) scheme and the newer NT hash. When a user logs on to a server from a remote workstation, the user is identified by the username, sent across the network in plaintext (no worries here; it’s not a secret anyway!). The server then generates a 64-bit random number and sends it to the client (also in plaintext). This number is the challenge.

Using the LanMan scheme, the client system then encrypts the challenge using DES. Recall that DES employs a 56-bit key, acts on a 64-bit block of data, and produces a 64-bit output. In this case, the 64-bit data block is the random number. The client actually uses three different DES keys to encrypt the random number, producing three different 64-bit outputs. The first key is the first seven bytes (56 bits) of the password’s hash value, the second key is the next seven bytes in the password’s hash, and the third key is the remaining two bytes of the password’s hash concatenated with five zero-filled bytes. (So, for the example above, the three DES keys would be 60771b22d73c34bd4a290a79c8b0, and 9f180000000000.) Each key is applied to the random number resulting in three 64-bit outputs, which comprise the response. Thus, the server’s 8-byte challenge yields a 24-byte response from the client and this is all that would be seen on the network. The server, for its part, does the same calculation to ensure that the values match.

There is, however, a significant weakness to this system. Specifically, the response is generated in such a way as to effectively reduce 16-byte hash to three smaller hashes, of length seven, seven, and two. Thus, a password cracker has to break at most a 7-byte hash. One Windows NT vulnerability test program that I have used in the past will report passwords that are “too short,” defined as “less than 8 characters.” When I asked how the program knew that passwords were too short, the software’s salespeople suggested to me that the program broke the passwords to determine their length. This is undoubtedly not true; all the software really has to do is look at the second 7-byte block and some known value indicates that it is empty, which would indicate a password of seven or less characters.

Consider the following example, showing the LanMan hash of two different short passwords (take a close look at the last 8 bytes):

AA: 89D42A44E77140AAAAD3B435B51404EE
AAA: 1C3A2B6D939A1021AAD3B435B51404EE

Note that the NT hash provides no such clue:

AA: C5663434F963BE79C8FD99F535E7AAD8
AAA: 6B6E0FB2ED246885B98586C73B5BFB77

It is worth noting that the discussion above describes the Microsoft version of CHAP, or MS-CHAP (MS-CHAPv2 is described in RFC 2759). MS-CHAP assumes that it is working with hashed values of the password as the key to encrypting the challenge. More traditional CHAP (RFC 1994) assumes that it is starting with passwords in plaintext. The relevance of this observation is that a CHAP client, for example, cannot be authenticated by an MS-CHAP server; both client and server must use the same CHAP version.

5.2. Some of the Finer Details of Diffie-Hellman

The first published public-key crypto algorithm was Diffie-Hellman. The mathematical “trick” of this scheme is that it is relatively easy to compute exponents compared to computing discrete logarithms. Diffie-Hellman allows two parties — the ubiquitous Alice and Bob — to generate a secret key; they need to exchange some information over an unsecure communications channel to perform the calculation but an eavesdropper cannot determine the shared key based upon this information.

Diffie-Hellman works like this. Alice and Bob start by agreeing on a large prime number, n. They also have to choose some number g so that g<n.

There is actually another constraint on g, specifically that it must be primitive with respect to n. Primitive is a definition that is a little beyond the scope of our discussion but basically g is primitive to n if we can find integers i so that gi = j mod n for all values of j from 1 to n-1. As an example, 2 is not primitive to 7 because the set of powers of 2 from 1 to 6, mod 7 = {2,4,1,2,4,1}. On the other hand, 3 is primitive to 7 because the set of powers of 3 from 1 to 6, mod 7 = {3,2,6,4,5,1}.

(The definition of primitive introduced a new term to some readers, namely mod. The phrase x mod y (and read as written!) means “take the remainder after dividing x by y.” Thus, 1 mod 7 = 1, 9 mod 6 = 3, and 8 mod 8 = 0.)

Anyway, either Alice or Bob selects n and g; they then tell the other party what the values are. Alice and Bob then work independently:

Alice…
Choose a large random number, x
Send to Bob: X = gx mod n
Compute: KA = Yx mod n
Bob…
Choose a large random number, y
Send to Alice: Y = gy mod n
Compute: KB = Xy mod n

Note that x and y are kept secret while X and Y are openly shared; these are the private and public keys, respectively. Based on their own private key and the public key learned from the other party, Alice and Bob have computed their secret keys, KA and KB, respectively, which are equal to gxy mod n.

Perhaps a small example will help here. Although Alice and Bob will really choose large values for n and g, I will use small values for example only; let’s use n=7 and g=3.

Alice…
Choose x=2
Send to Bob: X = 32 mod 7 = 2
KA = 62 mod 7 = 1
Bob…
Choose y=3
Send to Alice: Y = 33 mod 7 = 6
KB = 23 mod 7 = 1

In this example, then, Alice and Bob will both find the secret key 1 which is, indeed, 36 mod 7. If an eavesdropper (Mallory) was listening in on the information exchange between Alice and Bob, he would learn g, n, X, and Y which is a lot of information but insufficient to compromise the key; as long as x and y remain unknown, K is safe. As said above, calculating X as gx is a lot easier than finding x as logg X!

A short digression on modulo arithmetic. In the paragraph above, we noted that 36 mod 7 = 1. This can be confirmed, of course, by noting that:

36 = 729 = 104*7 + 1

There is a nice property of modulo arithmetic, however, that makes this determination a little easier, namely: (a mod x)(b mod x) = (ab mod x). Therefore, one possible shortcut is to note that 36 = (33)(33). Therefore, 36 mod 7 = (33 mod 7)(33 mod 7) = (27 mod 7)(27 mod 7) = 6*6 mod 7 = 36 mod 7 = 1.

Diffie-Hellman can also be used to allow key sharing amongst multiple users. Note again that the Diffie-Hellman algorithm is used to generate secret keys, not to encrypt and decrypt messages.

5.3. Some of the Finer Details of RSA Public-Key Cryptography

Unlike Diffie-Hellman, RSA can be used for key exchange as well as digital signatures and the encryption of small blocks of data. Today, RSA is primary used to encrypt the session key used for secret key encryption (message integrity) or the message’s hash value (digital signature). RSA’s mathematical hardness comes from the ease in calculating large numbers and the difficulty in finding the prime factors of those large numbers. Although employed with numbers using hundreds of digits, the math behind RSA is relatively straight-forward.

To create an RSA public/private key pair, here are the basic steps:

  1. Choose two prime numbers, p and q. From these numbers you can calculate the modulus, n = pq.
  2. Select a third number, e, that is relatively prime to (i.e., it does not divide evenly into) the product (p-1)(q-1). The number e is the public exponent.
  3. Calculate an integer d from the quotient (ed-1)/[(p-1)(q-1)]. The number d is the private exponent.

The public key is the number pair (n,e). Although these values are publicly known, it is computationally infeasible to determine d from n and e if p and q are large enough.

To encrypt a message, M, with the public key, create the ciphertext, C, using the equation:

      C = M

e

     mod n

The receiver then decrypts the ciphertext with the private key using the equation:

      M = C

d

     mod n

Now, this might look a bit complex and, indeed, the mathematics does take a lot of computer power given the large size of the numbers; since p and q may be 100 digits (decimal) or more, d and e will be about the same size and n may be over 200 digits. Nevertheless, a simple example may help. In this example, the values for p, q, e, and d are purposely chosen to be very small and the reader will see exactly how badly these values perform, but hopefully the algorithm will be adequately demonstrated:

  1. Select p=3 and q=5.
  2. The modulus n = pq = 15.
  3. The value e must be relatively prime to (p-1)(q-1) = (2)(4) = 8. Select e=11
  4. The value d must be chosen so that (ed-1)/[(p-1)(q-1)] is an integer. Thus, the value (11d-1)/[(2)(4)] = (11d-1)/8 must be an integer. Calculate one possible value, d=3.
  5. Let’s say we wish to send the string SECRET. For this example, we will convert the string to the decimal representation of the ASCII values of the characters, which would be 83 69 67 82 69 84.
  6. The sender encrypts each digit one at a time (we have to because the modulus is so small) using the public key value (e,n)=(11,15). Thus, each ciphertext character Ci = Mi11 mod 15. The input digit string 0x836967826984 will be transmitted as 0x2c696d286924.
  7. The receiver decrypts each digit using the private key value (d,n)=(3,15). Thus, each plaintext character Mi = Ci3 mod 15. The input digit string 0x2c696d286924 will be converted to 0x836967826984and, presumably, reassembled as the plaintext string SECRET.

Again, the example above uses small values for simplicity and, in fact, shows the weakness of small values; note that 4, 6, and 9 do not change when encrypted, and that the values 2 and 8 encrypt to 8 and 2, respectively. Nevertheless, this simple example demonstrates how RSA can be used to exchange information.

RSA keylengths of 512 and 768 bits are considered to be pretty weak. The minimum suggested RSA key is 1024 bits; 2048 and 3072 bits are even better.

As an aside, Adam Back (http://www.cypherspace.org/~adam/) wrote a two-line Perl script to implement RSA. It employs dc, an arbitrary precision arithmetic package that ships with most UNIX systems:

print pack"C*",split/\D+/,`echo "16iII*o\U@{$/=$z;[(pop,pop,unpack"H*",<>
)]}\EsMsKsN0[lN*1lK[d2%Sa2/d0<X+d*lMLa^*lN%0]dsXx++lMlN/dsM0<J]dsJxp"|dc`

5.4. Some of the Finer Details of DES, Breaking DES, and DES Variants

The Data Encryption Standard (DES) has been in use since the mid-1970s, adopted by the National Bureau of Standards (NBS) [now the National Institute for Standards and Technology (NIST)] as Federal Information Processing Standard 46 (FIPS 46-3) and by the American National Standards Institute (ANSI) as X3.92.

As mentioned earlier, DES uses the Data Encryption Algorithm (DEA), a secret key block-cipher employing a 56-bit key operating on 64-bit blocks. FIPS 81 describes four modes of DES operation: Electronic Codebook (ECB), Cipher Block Chaining (CBC), Cipher Feedback (CFB), and Output Feedback (OFB). Despite all of these options, ECB is the most commonly deployed mode of operation.

NIST finally declared DES obsolete in 2004, and withdrew FIPS 46-3, 74, and 81 (Federal Register, July 26, 2004, 69(142), 44509-44510). Although other block ciphers will replace DES, it is still interesting to see how DES encryption is performed; not only is it sort of neat, but DES was the first crypto scheme commonly seen in non-govermental applications and was the catalyst for modern “public” cryptography. DES remains in many products — and cryptography students and cryptographers will continue to study DES for years to come.

DES Operational Overview

DES uses a 56-bit key. In fact, the 56-bit key is divided into eight 7-bit blocks and an 8th odd parity bit is added to each block (i.e., a “0” or “1” is added to the block so that there are an odd number of 1 bits in each 8-bit block). By using the 8 parity bits for rudimentary error detection, a DES key is actually 64 bits in length for computational purposes (although it only has 56 bits worth of randomness, or entropy).

FIGURE 6: DES enciphering algorithm.

DES then acts on 64-bit blocks of the plaintext, invoking 16 rounds of permutations, swaps, and substitutes, as shown in Figure 6. The standard includes tables describing all of the selection, permutation, and expansion operations mentioned below; these aspects of the algorithm are not secrets. The basic DES steps are:

  1. The 64-bit block to be encrypted undergoes an initial permutation (IP), where each bit is moved to a new bit position; e.g., the 1st, 2nd, and 3rd bits are moved to the 58th, 50th, and 42nd position, respectively.
  2. The 64-bit permuted input is divided into two 32-bit blocks, called left and right, respectively. The initial values of the left and right blocks are denoted L0 and R0.
  3. There are then 16 rounds of operation on the L and R blocks. During each iteration (where n ranges from 1 to 16), the following formulae apply:
        L

    n

         = R

    n-1

        R

    n

         = L

    n-1

         XOR f(R

    n-1

        ,K

    n

      )

    At any given step in the process, then, the new L block value is merely taken from the prior R block value. The new R block is calculated by taking the bit-by-bit exclusive-OR (XOR) of the prior L block with the results of applying the DES cipher function, f, to the prior R block and Kn. (Kn is a 48-bit value derived from the 64-bit DES key. Each round uses a different 48 bits according to the standard’s Key Schedule algorithm.)The cipher function, f, combines the 32-bit R block value and the 48-bit subkey in the following way. First, the 32 bits in the R block are expanded to 48 bits by an expansion function (E); the extra 16 bits are found by repeating the bits in 16 predefined positions. The 48-bit expanded R-block is then ORed with the 48-bit subkey. The result is a 48-bit value that is then divided into eight 6-bit blocks. These are fed as input into 8 selection (S) boxes, denoted S1,…,S8. Each 6-bit input yields a 4-bit output using a table lookup based on the 64 possible inputs; this results in a 32-bit output from the S-box. The 32 bits are then rearranged by a permutation function (P), producing the results from the cipher function.

  4. The results from the final DES round — i.e., L16 and R16 — are recombined into a 64-bit value and fed into an inverse initial permutation (IP-1). At this step, the bits are rearranged into their original positions, so that the 58th, 50th, and 42nd bits, for example, are moved back into the 1st, 2nd, and 3rd positions, respectively. The output from IP-1 is the 64-bit ciphertext block.

Consider this example with the given 56-bit key and input:

      Key:

1100101 0100100 1001001 0011101 0110101 0101011 1101100 0011010

      Input character string:

 GoAggies

      Input bit string:

 11100010 11110110 10000010 11100110 11100110 10010110 10100110 11001110

      Output bit string:

10011111 11110010 10000000 10000001 01011011 00101001 00000011 00101111

      Output character string:

ùOÚ”ÀôBreaking DES

The mainstream cryptographic community has long held that DES’s 56-bit key was too short to withstand a brute-force attack from modern computers. Remember Moore’s Law: computer power doubles every 18 months. Given that increase in power, a key that could withstand a brute-force guessing attack in 1975 could hardly be expected to withstand the same attack a quarter century later.

DES is even more vulnerable to a brute-force attack because it is often used to encrypt words, meaning that the entropy of the 64-bit block is, effectively, greatly reduced. That is, if we are encrypting random bit streams, then a given byte might contain any one of 28 (256) possible values and the entire 64-bit block has 264, or about 18.5 quintillion, possible values. If we are encrypting words, however, we are most likely to find a limited set of bit patterns; perhaps 70 or so if we account for upper and lower case letters, the numbers, space, and some punctuation. This means that only about ¼ of the bit combinations of a given byte are likely to occur.

Despite this criticism, the U.S. government insisted throughout the mid-1990s that 56-bit DES was secure and virtually unbreakable if appropriate precautions were taken. In response, RSA Laboratories sponsored a series of cryptographic challenges to prove that DES was no longer appropriate for use.

DES Challenge I was launched in March 1997. It was completed in 84 days by R. Verser in a collaborative effort using thousands of computers on the Internet.

The first DES II challenge lasted 40 days in early 1998. This problem was solved by distributed.net, a worldwide distributed computing network using the spare CPU cycles of computers around the Internet (participants in distributed.net’s activities load a client program that runs in the background, conceptually similar to the SETI @Home “Search for Extraterrestrial Intelligence” project). The distributed.net systems were checking 28 billion keys per second by the end of the project.

The second DES II challenge lasted less than 3 days. On July 17, 1998, the Electronic Frontier Foundation (EFF) announced the construction of hardware that could brute-force a DES key in an average of 4.5 days. Called Deep Crack, the device could check 90 billion keys per second and cost only about $220,000 including design (it was erroneously and widely reported that subsequent devices could be built for as little as $50,000). Since the design is scalable, this suggests that an organization could build a DES cracker that could break 56-bit keys in an average of a day for as little as $1,000,000. Information about the hardware design and all software can be obtained from the EFF.

The DES III challenge, launched in January 1999, was broken is less than a day by the combined efforts of Deep Crack and distributed.net. This is widely considered to have been the final nail in DES’s coffin.

The Deep Crack algorithm is actually quite interesting. The general approach that the DES Cracker Project took was not to break the algorithm mathematically but instead to launch a brute-force attack by guessing every possible key. A 56-bit key yields 256, or about 72 quadrillion, possible values. So the DES cracker team looked for any shortcuts they could find! First, they assumed that some recognizable plaintext would appear in the decrypted string even though they didn’t have a specific known plaintext block. They then applied all 256 possible key values to the 64-bit block (I don’t mean to make this sound simple!). The system checked to see if the decrypted value of the block was “interesting,” which they defined as bytes containing one of the alphanumeric characters, space, or some punctuation. Since the likelihood of a single byte being “interesting” is about ¼, then the likelihood of the entire 8-byte stream being “interesting” is about ¼8, or 1/65536 (½16). This dropped the number of possible keys that might yield positive results to about 240, or about a trillion.

They then made the assumption that an “interesting” 8-byte block would be followed by another “interesting” block. So, if the first block of ciphertext decrypted to something interesting, they decrypted the next block; otherwise, they abandoned this key. Only if the second block was also “interesting” did they examine the key closer. Looking for 16 consecutive bytes that were “interesting” meant that only 224, or 16 million, keys needed to be examined further. This further examination was primarily to see if the text made any sense. Note that possible “interesting” blocks might be 1hJ5&aB7 or DEPOSITS; the latter is more likely to produce a better result. And even a slow laptop today can search through lists of only a few million items in a relatively short period of time. (Interested readers are urged to read Cracking DESand EFF’s Cracking DES page.)

It is well beyond the scope of this paper to discuss other forms of breaking DES and other codes. Nevertheless, it is worth mentioning a couple of forms of cryptanalysis that have been shown to be effective against DES. Differential cryptanalysis, invented in 1990 by E. Biham and A. Shamir (of RSA fame), is a chosen-plaintext attack. By selecting pairs of plaintext with particular differences, the cryptanalyst examines the differences in the resultant ciphertext pairs. Linear plaintext, invented by M. Matsui, uses a linear approximation to analyze the actions of a block cipher (including DES). Both of these attacks can be more efficient than brute force.

DES Variants

Once DES was “officially” broken, several variants appeared. But none of them came overnight; work at hardening DES had already been underway. In the early 1990s, there was a proposal to increase the security of DES by effectively increasing the key length by using multiple keys with multiple passes. But for this scheme to work, it had to first be shown that the DES function is not a group, as defined in mathematics. If DES was a group, then we could show that for two DES keys, X1 and X2, applied to some plaintext (P), we can find a single equivalent key, X3, that would provide the same result; i.e.,:

EX2(EX1(P)) = EX3(P)

where EX(P) represents DES encryption of some plaintext P using DES key X. If DES were a group, it wouldn’t matter how many keys and passes we applied to some plaintext; we could always find a single 56-bit key that would provide the same result.

As it happens, DES was proven to not be a group so that as we apply additional keys and passes, the effective key length increases. One obvious choice, then, might be to use two keys and two passes, yielding an effective key length of 112 bits. Let’s call this Double-DES. The two keys, Y1 and Y2, might be applied as follows:

C = EY2(EY1(P))
P = DY1(DY2(C))

where EY(P) and DY(C) represent DES encryption and decryption, respectively, of some plaintext P and ciphertext C, respectively, using DES key Y.

So far, so good. But there’s an interesting attack that can be launched against this “Double-DES” scheme. First, notice that the applications of the formula above can be thought of with the following individual steps (where C’ and P’ are intermediate results):

C’ = EY1(P) and C = EY2(C’)
P’ = DY2(C) and P = DY1(P’)

Unfortunately, C’=P’. That leaves us vulnerable to a simple known plaintext attack (sometimes called “Meet-in-the-middle”) where the attacker knows some plaintext (P) and its matching ciphertext (C). To obtain C’, the attacker needs to try all 256 possible values of Y1 applied to P; to obtain P’, the attacker needs to try all 256 possible values of Y2 applied to C. Since C’=P’, the attacker knows when a match has been achieved — after only 256 + 256 = 257 key searches, only twice the work of brute-forcing DES. So “Double-DES” won’t work.

Triple-DES (3DES), based upon the Triple Data Encryption Algorithm (TDEA), is described in FIPS 46-3. 3DES, which is not susceptible to a meet-in-the-middle attack, employs three DES passes and one, two, or three keys called K1, K2, and K3. Generation of the ciphertext (C) from a block of plaintext (P) is accomplished by:

C = EK3(DK2(EK1(P)))

where EK(P) and DK(P) represent DES encryption and decryption, respectively, of some plaintext P using DES key K. (For obvious reasons, this is sometimes referred to as an encrypt-decrypt-encrypt mode operation.)

Decryption of the ciphertext into plaintext is accomplished by:

P = DK1(EK2(DK3(C)))

The use of three, independent 56-bit keys provides 3DES with an effective key length of 168 bits. The specification also defines use of two keys where, in the operations above, K3 = K1; this provides an effective key length of 112 bits. Finally, a third keying option is to use a single key, so that K3 = K2 = K1 (in this case, the effective key length is 56 bits and 3DES applied to some plaintext, P, will yield the same ciphertext, C, as normal DES would with that same key). Given the relatively low cost of key storage and the modest increase in processing due to the use of longer keys, the best recommended practices are that 3DES be employed with three keys.

Another variant of DES, called DESX, is due to Ron Rivest. Developed in 1996, DESX is a very simple algorithm that greatly increases DES’s resistance to brute-force attacks without increasing its computational complexity. In DESX, the plaintext input is XORed with 64 additional key bits prior to encryption and the output is likewise XORed with the 64 key bits. By adding just two XOR operations, DESX has an effective keylength of 120 bits against an exhaustive key-search attack. As it happens, DESX is no more immune to other types of more sophisticated attacks, such as differential or linear cryptanalysis, but brute-force is the primary attack vector on DES.

Closing Comments

Although DES has been deprecated and replaced by the Advanced Encryption Standard (AES) because of its vulnerability to a modestly-priced brute-force attack, many applications continue to rely on DES for security, and many software designers and implementers continue to include DES in new applications. In some cases, use of DES is wholly appropriate but, in general, DES should not continue to be promulgated in production software and hardware. RFC 4772 discusses the security implications of employing DES.

On a final note, readers may be interested in seeing The Illustrated DES Spreadsheet (J. Hughes, 2004), an Excel implementation of DES.

5.5. Pretty Good Privacy (PGP)

Pretty Good Privacy (PGP) is one of today’s most widely used public key cryptography programs. Developed by Philip Zimmermann in the early 1990s and long the subject of controversy, PGP is available as a plug-in for many e-mail clients, such as Claris Emailer, Microsoft Outlook/Outlook Express, and Qualcomm Eudora.

PGP can be used to sign or encrypt e-mail messages with the mere click of the mouse. Depending upon the version of PGP, the software uses SHA or MD5 for calculating the message hash; CAST, Triple-DES, or IDEA for encryption; and RSA or DSS/Diffie-Hellman for key exchange and digital signatures.

When PGP is first installed, the user has to create a key-pair. One key, the public key, can be advertised and widely circulated. The private key is protected by use of a passphrase. The passphrase has to be entered every time the user accesses their private key.

 -----BEGIN PGP SIGNED MESSAGE-----
 Hash: SHA1

 Hi Carol.

 What was that pithy Groucho Marx quote?

 /kess

 -----BEGIN PGP SIGNATURE-----
 Version: PGP for Personal Privacy 5.0
 Charset: noconv

 iQA/AwUBNFUdO5WOcz5SFtuEEQJx/ACaAgR97+vvDU6XWELV/GANjAAgBtUAnjG3
 Sdfw2JgmZIOLNjFe7jP0Y8/M
 =jUAU
 -----END PGP SIGNATURE-----

 

FIGURE 7: A PGP signed message. The sender uses their private key; at the destination, the sender’s e-mail address yields the public key from the receiver’s keyring.

Figure 7 shows a PGP signed message. This message will not be kept secret from an eavesdropper, but a recipient can be assured that the message has not been altered from what the sender transmitted. In this instance, the sender signs the message using their own private key. The receiver uses the sender’s public key to verify the signature; the public key is taken from the receiver’s keyring based on the sender’s e-mail address. Note that the signature process does not work unless the sender’s public key is on the receiver’s keyring.

-----BEGIN PGP MESSAGE-----
Version: PGP for Personal Privacy 5.0
MessageID: DAdVB3wzpBr3YRunZwYvhK5gBKBXOb/m

qANQR1DBwU4D/TlT68XXuiUQCADfj2o4b4aFYBcWumA7hR1Wvz9rbv2BR6WbEUsy
ZBIEFtjyqCd96qF38sp9IQiJIKlNaZfx2GLRWikPZwchUXxB+AA5+lqsG/ELBvRa
c9XefaYpbbAZ6z6LkOQ+eE0XASe7aEEPfdxvZZT37dVyiyxuBBRYNLN8Bphdr2zv
z/9Ak4/OLnLiJRk05/2UNE5Z0a+3lcvITMmfGajvRhkXqocavPOKiin3hv7+Vx88
uLLem2/fQHZhGcQvkqZVqXx8SmNw5gzuvwjV1WHj9muDGBY0MkjiZIRI7azWnoU9
3KCnmpR60VO4rDRAS5uGl9fioSvze+q8XqxubaNsgdKkoD+tB/4u4c4tznLfw1L2
YBS+dzFDw5desMFSo7JkecAS4NB9jAu9K+f7PTAsesCBNETDd49BTOFFTWWavAfE
gLYcPrcn4s3EriUgvL3OzPR4P1chNu6sa3ZJkTBbriDoA3VpnqG3hxqfNyOlqAka
mJJuQ53Ob9ThaFH8YcE/VqUFdw+bQtrAJ6NpjIxi/x0FfOInhC/bBw7pDLXBFNaX
HdlLQRPQdrmnWskKznOSarxq4GjpRTQo4hpCRJJ5aU7tZO9HPTZXFG6iRIT0wa47

AR5nvkEKoIAjW5HaDKiJriuWLdtN4OXecWvxFsjR32ebz76U8aLpAK87GZEyTzBx
dV+lH0hwyT/y1cZQ/E5USePP4oKWF4uqquPee1OPeFMBo4CvuGyhZXD/18Ft/53Y
WIebvdiCqsOoabK3jEfdGExce63zDI0=
=MpRf
-----END PGP MESSAGE-----

 

FIGURE 8: A PGP encrypted message. The receiver’s e-mail address is the pointer to the public key in the sender’s keyring. At the destination side, the receiver uses their own private key.

Figure 8 shows a PGP encrypted message (PGP compresses the file, where practical, prior to encryption because encrypted files lose their randomness and, therefore, cannot be compressed). In this case, public key methods are used to exchange the session key for the actual message encryption using secret-key cryptography. In this case, the receiver’s e-mail address is the pointer to the public key in the sender’s keyring; in fact, the same message can be sent to multiple recipients and the message will not be significantly longer since all that needs to be added is the session key encrypted by each receiver’s private key. When the message is received, the recipient must use their private key to extract the session secret key to successfully decrypt the message (Figure 9).

 Hi Gary,

 "Outside of a dog, a book is man's best friend.
 Inside of a dog, it's too dark to read."

 Carol

 

FIGURE 9: The decrypted message.

It is worth noting that PGP was one of the first so-called “hybrid cryptosystems” that combined aspects of SKC and PKC. When Zimmermann was first designing PGP in the late-1980s, he wanted to use RSA to encrypt the entire message. The PCs of the days, however, suffered significant performance degradation when executing RSA so he hit upon the idea of using SKC to encrypt the message and PKC to encrypt the SKC key.

PGP went into a state of flux in 2002. Zimmermann sold PGP to Network Associates, Inc. (NAI) in 1997 and himself resigned from NAI in early 2001. In March 2002, NAI announced that they were dropping support for the commercial version of PGP having failed to find a buyer for the product willing to pay what NAI wanted. In August 2002, PGP was purchased from NAI by PGP Corp. (http://www.pgp.com/). Meanwhile, there are many freeware versions of PGP available through the International PGP Page and the OpenPGP Alliance. Also check out the GNU Privacy Guard (GnuPG), a GNU project implementation of OpenPGP (defined in RFC 2440).

5.6. IP Security (IPsec) Protocol

NOTE: The information in this section assumes that the reader is familiar with the Internet Protocol (IP), at least to the extent of the packet format and header contents. More information about IP can be found in An Overview of TCP/IP Protocols and the Internet. More information about IPv6 can be found in IPv6: The Next Generation Internet Protocol.

The Internet and the TCP/IP protocol suite were not built with security in mind. This statement is not meant as a criticism; the baseline UDP, TCP, IP, and ICMP protocols were written in 1980 and built for the relatively closed ARPANET community. TCP/IP wasn’t designed for the commercial-grade financial transactions that they now see nor for virtual private networks (VPNs) on the Internet. To bring TCP/IP up to today’s security necessities, the Internet Engineering Task Force (IETF) formed the IP Security Protocol Working Group which, in turn, developed the IP Security (IPsec) protocol. IPsec is not a single protocol, in fact, but a suite of protocols providing a mechanism to provide data integrity, authentication, privacy, and nonrepudiation for the classic Internet Protocol (IP). Although intended primarily for IP version 6 (IPv6), IPsec can also be employed by the current version of IP, namely IP version 4 (IPv4).

As shown in Table 3, IPsec is described in nearly a dozen RFCs. RFC 4301, in particular, describes the overall IP security architecture and RFC 2411 provides an overview of the IPsec protocol suite and the documents describing it.

IPsec can provide either message authentication and/or encryption. The latter requires more processing than the former, but will probably end up being the preferred usage for applications such as VPNs and secure electronic commerce.

Central to IPsec is the concept of a security association (SA). Authentication and confidentiality using AH or ESP use SAs and a primary role of IPsec key exchange it to establish and maintain SAs. An SA is a simplex (one-way or unidirectional) logical connection between two communicating IP endpoints that provides security services to the traffic carried by it using either AH or ESP procedures. The endpoint of an SA can be an IP host or IP security gateway (e.g., a proxy server, VPN server, etc.). Providing security to the more typical scenario of two-way (bi-directional) communication between two endpoints requires the establishment of two SAs (one in each direction).

An SA is uniquely identified by a 3-tuple composed of:

  • Security Parameter Index (SPI), a 32-bit identifier of the connection
  • IP Destination Address
  • security protocol (AH or ESP) identifier

The IP Authentication Header (AH), described in RFC 4302, provides a mechanism for data integrity and data origin authentication for IP packets using HMAC with MD5 (RFC 2403), HMAC with SHA-1 (RFC 2404), or HMAC with RIPEMD (RFC 2857). See also RFC 4305.

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | Next Header   |  Payload Len  |          RESERVED             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                 Security Parameters Index (SPI)               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    Sequence Number Field                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   +                Integrity Check Value-ICV (variable)           |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

 

FIGURE 10: IPsec Authentication Header format. (From RFC 4302)

Figure 10 shows the format of the IPsec AH. The AH is merely an additional header in a packet, more or less representing another protocol layer above IP (this is shown in Figure 12 below). Use of the IP AH is indicated by placing the value 51 (0x33) in the IPv4 Protocol or IPv6 Next Header field in the IP packet header. The AH follows mandatory IPv4/IPv6 header fields and precedes higher layer protocol (e.g., TCP, UDP) information. The contents of the AH are:

  • Next Header: An 8-bit field that identifies the type of the next payload after the Authentication Header.
  • Payload Length: An 8-bit field that indicates the length of AH in 32-bit words (4-byte blocks), minus “2”. [The rationale for this is somewhat counter intuitive but technically important. All IPv6 extension headers encode the header extension length (Hdr Ext Len) field by first subtracting 1 from the header length, which is measured in 64-bit words. Since AH was originally developed for IPv6, it is an IPv6 extension header. Since its length is measured in 32-bit words, however, the Payload Length is calculated by subtracting 2 (32 bit words) to maintain consistency with IPv6 coding rules.] In the default case, the three 32-bit word fixed portion of the AH is followed by a 96-bit authentication value, so the Payload Length field value would be 4.
  • Reserved: This 16-bit field is reserved for future use and always filled with zeros.
  • Security Parameters Index (SPI): An arbitrary 32-bit value that, in combination with the destination IP address and security protocol, uniquely identifies the Security Association for this datagram. The value 0 is reserved for local, implementation-specific uses and values between 1-255 are reserved by the Internet Assigned Numbers Authority (IANA) for future use.
  • Sequence Number: A 32-bit field containing a sequence number for each datagram; initially set to 0 at the establishment of an SA. AH uses sequence numbers as an anti-replay mechanism, to prevent a “person-in-the-middle” attack. If anti-replay is enabled (the default), the transmitted Sequence Number is never allowed to cycle back to 0; therefore, the sequence number must be reset to 0 by establishing a new SA prior to the transmission of the 232nd packet.
  • Authentication Data: A variable-length, 32-bit aligned field containing the Integrity Check Value (ICV) for this packet (default length = 96 bits). The ICV is computed using the authentication algorithm specified by the SA, such as DES, MD5, or SHA-1. Other algorithms may also be supported.

The IP Encapsulating Security Payload (ESP), described in RFC 4303, provides message integrity and privacy mechanisms in addition to authentication. As in AH, ESP uses HMAC with MD5, SHA-1, or RIPEMD authentication (RFC 2403/RFC 2404/RFC 2857); privacy is provided using DES-CBC encryption (RFC 2405), NULL encryption (RFC 2410), other CBC-mode algorithms (RFC 2451), or AES (RFC 3686). See also RFC 4305 and RFC 4308.

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ----
   |               Security Parameters Index (SPI)                 | ^Int.
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Cov-
   |                      Sequence Number                          | |ered
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ----  
   |                    Payload Data* (variable)                   | |   ^
   ~                                                               ~ |   |
   |                                                               | |Conf.
   +               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Cov-
   |               |     Padding (0-255 bytes)                     | |ered*
   +-+-+-+-+-+-+-+-+               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |   |
   |                               |  Pad Length   | Next Header   | v   v
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ------
   |         Integrity Check Value-ICV   (variable)                |
   ~                                                               ~
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

       * If included in the Payload field, cryptographic synchronization
         data, e.g., an Initialization Vector (IV), usually is not
         encrypted per se, although it often is referred to as being
         being part of the ciphertext.

 

FIGURE 11: IPsec Encapsulating Security Payload format. (From RFC 4303)

Figure 11 shows the format of the IPsec ESP information. Use of the IP ESP format is indicated by placing the value 50 (0x32) in the IPv4 Protocol or IPv6 Next Header field in the IP packet header. The ESP header (i.e., SPI and sequence number) follows mandatory IPv4/IPv6 header fields and precedes higher layer protocol (e.g., TCP, UDP) information. The contents of the ESP packet are:

  • Security Parameters Index: (see description for this field in the AH, above.)
  • Sequence Number: (see description for this field in the AH, above.)
  • Payload Data: A variable-length field containing data as described by the Next Header field. The contents of this field could be encrypted higher layer data or an encrypted IP packet.
  • Padding: Between 0 and 255 octets of padding may be added to the ESP packet. There are several applications that might use the padding field. First, the encryption algorithm that is used may require that the plaintext be a multiple of some number of bytes, such as the block size of a block cipher; in this case, the Padding field is used to fill the plaintext to the size required by the algorithm. Second, padding may be required to ensure that the ESP packet and resulting ciphertext terminate on a 4-byte boundary. Third, padding may be used to conceal the actual length of the payload. Unless another value is specified by the encryption algorithm, the Padding octets take on the value 1, 2, 3, … starting with the first Padding octet. This scheme is used because, in addition to being simple to implement, it provides some protection against certain forms of “cut and paste” attacks.
  • Pad Length: An 8-bit field indicating the number of bytes in the Padding field; contains a value between 0-255.
  • Next Header: An 8-bit field that identifies the type of data in the Payload Data field, such as an IPv6 extension header or a higher layer protocol identifier.
  • Authentication Data: (see description for this field in the AH, above.)

Two types of SAs are defined in IPsec, regardless of whether AH or ESP is employed. A transport mode SA is a security association between two hosts. Transport mode provides the authentication and/or encryption service to the higher layer protocol. This mode of operation is only supported by IPsec hosts. A tunnel mode SA is a security association applied to an IP tunnel. In this mode, there is an “outer” IP header that specifies the IPsec destination and an “inner” IP header that specifies the destination for the IP packet. This mode of operation is supported by both hosts and security gateways.

  ORIGINAL PACKET BEFORE APPLYING AH

         ----------------------------
   IPv4  |orig IP hdr  |     |      |
         |(any options)| TCP | Data |
         ----------------------------

         ---------------------------------------
   IPv6  |             | ext hdrs |     |      |
         | orig IP hdr |if present| TCP | Data |
         ---------------------------------------

  AFTER APPLYING AH (TRANSPORT MODE)

          -------------------------------------------------------

    IPv4  |original IP hdr (any options) | AH | TCP |    Data   |
          -------------------------------------------------------
          |<- mutable field processing ->|<- immutable fields ->|
          |<----- authenticated except for mutable fields ----->|

         ------------------------------------------------------------
   IPv6  |             |hop-by-hop, dest*, |    | dest |     |      |
         |orig IP hdr  |routing, fragment. | AH | opt* | TCP | Data |
         ------------------------------------------------------------
         |<--- mutable field processing -->|<-- immutable fields -->|
         |<---- authenticated except for mutable fields ----------->|

               * = if present, could be before AH, after AH, or both

  AFTER APPLYING AH (TUNNEL MODE)

        ----------------------------------------------------------------
   IPv4 |                              |    | orig IP hdr*  |   |      |
        |new IP header * (any options) | AH | (any options) |TCP| Data |
        ----------------------------------------------------------------
        |<- mutable field processing ->|<------ immutable fields ----->|
        |<- authenticated except for mutable fields in the new IP hdr->|

        --------------------------------------------------------------
   IPv6 |           | ext hdrs*|    |            | ext hdrs*|   |    |
        |new IP hdr*|if present| AH |orig IP hdr*|if present|TCP|Data|
        --------------------------------------------------------------
        |<--- mutable field -->|<--------- immutable fields -------->|
        |       processing     |
        |<-- authenticated except for mutable fields in new IP hdr ->|

          * = if present, construction of outer IP hdr/extensions and
              modification of inner IP hdr/extensions is discussed in
              the Security Architecture document.

 

FIGURE 12: IPsec tunnel and transport modes for AH. (Adapted from RFC 4302)

Figure 12 show the IPv4 and IPv6 packet formats when using AH in both transport and tunnel modes. Initially, an IPv4 packet contains a normal IPv4 header (which may contain IP options), followed by the higher layer protocol header (e.g., TCP or UDP), followed by the higher layer data itself. An IPv6 packet is similar except that the packet starts with the mandatory IPv6 header followed by any IPv6 extension headers, and then followed by the higher layer data.

Note that in both transport and tunnel modes, the entire IP packet is covered by the authentication except for the mutable fields. A field is mutable if its value might change during transit in the network; IPv4 mutable fields include the fragment offset, time to live, and checksum fields. Note, in particular, that the address fields are not mutable.

    ORIGINAL PACKET BEFORE APPLYING ESP

            ----------------------------
      IPv4  |orig IP hdr  |     |      |
            |(any options)| TCP | Data |
            ----------------------------

            ---------------------------------------
      IPv6  |             | ext hdrs |     |      |
            | orig IP hdr |if present| TCP | Data |
            ---------------------------------------

    AFTER APPLYING ESP (TRANSPORT MODE)

            -------------------------------------------------
      IPv4  |orig IP hdr  | ESP |     |      |   ESP   | ESP|
            |(any options)| Hdr | TCP | Data | Trailer | ICV|
            -------------------------------------------------
                                |<---- encryption ---->|
                          |<-------- integrity ------->|

            ---------------------------------------------------------
      IPv6  | orig |hop-by-hop,dest*,|   |dest|   |    | ESP   | ESP|

            |IP hdr|routing,fragment.|ESP|opt*|TCP|Data|Trailer| ICV|
            ---------------------------------------------------------
                                         |<--- encryption ---->|
                                     |<------ integrity ------>|

                * = if present, could be before ESP, after ESP, or both

    AFTER APPLYING ESP (TUNNEL MODE)

            -----------------------------------------------------------
      IPv4  | new IP hdr+ |     | orig IP hdr+  |   |    | ESP   | ESP|
            |(any options)| ESP | (any options) |TCP|Data|Trailer| ICV|

            -----------------------------------------------------------
                                |<--------- encryption --------->|
                          |<------------- integrity ------------>|

            ------------------------------------------------------------
      IPv6  | new+ |new ext |   | orig+|orig ext |   |    | ESP   | ESP|
            |IP hdr| hdrs+  |ESP|IP hdr| hdrs+   |TCP|Data|Trailer| ICV|
            ------------------------------------------------------------
                                |<--------- encryption ---------->|
                            |<------------ integrity ------------>|

            + = if present, construction of outer IP hdr/extensions and
                modification of inner IP hdr/extensions is discussed in
                the Security Architecture document.

 

FIGURE 13: IPsec tunnel and transport modes for ESP. (Adapted from RFC 4303)

Figure 13 shows the IPv4 and IPv6 packet formats when using ESP in both transport and tunnel modes.

  • As with AH, we start with a standard IPv4 or IPv6 packet.
  • In transport mode, the higher layer header and data, as well as ESP trailer information, is encrypted and the entire ESP packet is authenticated. In the case of IPv6, some of the IPv6 extension options can precede or follow the ESP header.
  • In tunnel mode, the original IP packet is encrypted and placed inside of an “outer” IP packet, while the entire ESP packet is authenticated.

Note a significant difference in the scope of ESP and AH. AH authenticates the entire packet transmitted on the network whereas ESP only covers a portion of the packet transmitted on the network (the higher layer data in transport mode and the entire original packet in tunnel mode). The reason for this is straight-forward; in AH, the authentication data for the transmission fits neatly into an additional header whereas ESP creates an entirely new packet which is the one encrypted and/or authenticated. But the ramifications are significant. ESP transport mode as well as AH in both modes protect the IP address fields of the original transmissions. Thus, using IPsec in conjunction with network address translation (NAT) might be problematic because NAT changes the values of these fields after IPsec processing.

The third component of IPsec is the establishment of security associations and key management. These tasks can be accomplished in one of two ways.

The simplest form of SA and key management is manual management. In this method, a security administer or other individual manually configures each system with the key and SA management data necessary for secure communication with other systems. Manual techniques are practical for small, reasonably static environments but they do not scale well.

For successful deployment of IPsec, however, a scalable, automated SA/key management scheme is necessary. Several protocols have defined for these functions:

  • The Internet Security Association and Key Management Protocol (ISAKMP) defines procedures and packet formats to establish, negotiate, modify and delete security associations, and provides the framework for exchanging information about authentication and key management (RFC 2407/RFC 2408). ISAKMP’s security association and key management is totally separate from key exchange.
  • The OAKLEY Key Determination Protocol (RFC 2412) describes a scheme by which two authenticated parties can exchange key information. OAKLEY uses the Diffie-Hellman key exchange algorithm.
  • The Internet Key Exchange (IKE) algorithm (RFC 2409) is the default automated key management protocol for IPsec.
  • An alternative to IKE is Photuris (RFC 2522/RFC 2523), a scheme for establishing short-lived session-keys between two authenticated parties without passing the session-keys across the Internet. IKE typically creates keys that may have very long lifetimes.

On a final note, IPsec authentication for both AH and ESP uses a scheme called HMAC, a keyed-hashing message authentication code described in FIPS 198 and RFC 2104. HMAC uses a shared secret key between two parties rather than public key methods for message authentication. The generic HMAC procedure can be used with just about any hash algorithm, although IPsec specifies support for at least MD5 and SHA-1 because of their widespread use.

In HMAC, both parties share a secret key. The secret key will be employed with the hash algorithm in a way that provides mutual authentication without transmitting the key on the line. IPsec key management procedures will be used to manage key exchange between the two parties.

Recall that hash functions operate on a fixed-size block of input at one time; MD5 and SHA-1, for example, work on 64 byte blocks. These functions then generate a fixed-size hash value; MD5 and SHA-1, in particular, produce 16 byte (128 bit) and 20 byte (160 bit) output strings, respectively. For use with HMAC, the secret key (K) should be at least as long as the hash output.

The following steps provide a simplified, although reasonably accurate, description of how the HMAC scheme would work with a particular plaintext MESSAGE:

  1. Pad K so that it is as long as an input block; call this padded key Kp.
  2. Compute the hash of the padded key followed by the message, i.e., HASH (Kp:MESSAGE).
  3. Transmit MESSAGE and the hash value.
  4. The receiver does the same procedure to pad K to create Kp.
  5. The receiver computes HASH (Kp:MESSAGE).
  6. The receiver compares the computed hash value with the received hash value. If they match, then the sender must know the secret key and the message is authenticated.

 

5.7. The SSL “Family” of Secure Transaction Protocols for the World Wide Web

The Secure Sockets Layer (SSL) protocol was developed by Netscape Communications to provide application-independent secure communication over the Internet for protocols such as the Hypertext Transfer Protocol (HTTP). SSL employs RSA and X.509 certificates during an initial handshake used to authenticate the server (client authentication is optional). The client and server then agree upon an encryption scheme; SSL v2 supported RC2 and RC4 with 40-bit keys, while SSL v3 adds support for DES, RC4 with a 128-bit key, and 3DES with a 168-bit key, all along with either MD5 or SHA-1 message hashes.

FIGURE 14: Browser encryption configuration screen (Firefox).

In 1997, SSL v3 was found to be breakable. By this time, the Internet Engineering Task Force (IETF) had already started work on a new, non-proprietary protocol called Transport Layer Security (TLS), described in RFC 2246. TLS extends SSL and supports additional crypto schemes, such as Diffie-Hellman key exchange and DSS digital signatures; RFC 4279 describes the pre-shared key crypto schemes supported by TLS. TLS is backward compatible with SSL (and, in fact, is recognized as SSL v3.1). SSL v3.0 and TLS v1.0 are the commonly supported versions on servers and browsers today (Figure 14); SSL v2.0 is rarely found today and, in fact, RFC 6176-compliant client and servers that support TLS will never negotiate the use of SSL v2.

                       CLIENT SERVER
 (using URL of form https://)       (listening on port 443) 

                  ClientHello ---->

                                    ServerHello
                                    Certificate*
                                    ServerKeyExchange*
                                    CertificateRequest*
                              <---- ServerHelloDone

                 Certificate*
            ClientKeyExchange
            CertifcateVerify*
           [ChangeCipherSpec]
                     Finished ---->

                                    [ChangeCipherSpec]
                              <---- Finished

             Application Data <---> Application Data

* Optional or situation-dependent messages;
  not always sent

                                     Adapted from RFC 2246

FIGURE 15: SSL/TLS protocol handshake.

Figure 15 shows the basic TLS (and SSL) message exchanges:

  1. URLs specifying the protocol https:// are directed to HTTP servers secured using SSL/TLS. The client will automatically try to make a TCP connection to the server at port 443. The client initiates the secure connection by sending a ClientHello message containing a Session identifier, highest SSL version number supported by the client, and lists of supported crypto and compression schemes (in preference order).
  2. The server examines the Session ID and if it is still in the server’s cache, it will attempt to re-establish a previous session with this client. If the Session ID is not recognized, the server will continue with the handshake to establish a secure session by responding with a ServerHello message. The ServerHello repeats the Session ID, indicates the SSL version to use for this connection (which will be the highest SSL version supported by the server and client), and specifies which encryption method and compression method to be used for this connection.
  3. There are a number of other optional messages that the server might send, including:
    • Certificate, which carries the server’s X.509 public key certificate (and, generally, the server’s public key). This message will always be sent unless the client and server have already agreed upon some form of anonymous key exchange. (This message is normally sent.)
    • ServerKeyExchange, which will carry a premaster secret when the server’s Certificate message does not contain enough data for this purpose; used in some key exchange schemes.
    • CertificateRequest, used to request the client’s certificate in those scenarios where client authentication is performed.
    • ServerHelloDone, indicating that the server has completed its portion of the key exchange handshake.
  4. The client now responds with a series of mandatory and optional messages:
    • Certificate, contains the client’s public key certificate when it has been requested by the server.
    • ClientKeyExchange, which usually carries the secret key to be used with the secret key crypto scheme.
    • CertificateVerify, used to provide explicit verification of a client’s certificate if the server is authenticating the client.
  5. TLS includes the change cipher spec protocol to indicate changes in the encryption method. This protocol contains a single message, ChangeCipherSpec, which is encrypted and compressed using the current (rather than the new) encryption and compression schemes. The ChangeCipherSpec message is sent by both client and server to notify the other station that all following information will employ the newly negotiated cipher spec and keys.
  6. The Finished message is sent after a ChangeCipherSpec message to confirm that the key exchange and authentication processes were successful.
  7. At this point, both client and server can exchange application data using the session encryption and compression schemes.Side Note: It would probably be helpful to make some mention of SSL as it is used today. Most of us have used SSL to engage in a secure, private transaction with some vendor. The steps are something like this. During the SSL exchange with the vendor’s secure server, the server sends its certificate to our client software. The certificate includes the vendor’s public key and a signature from the CA that issued the vendor’s certificate. Our browser software is shipped with the major CAs’ certificates which contains their public key; in that way we authenticate the server. Note that the server doesnot use a certificate to authenticate us! Instead, we are generally authenticated when we provide our credit card number; the server checks to see if the card purchase will be authorized by the credit card company and, if so, considers us valid and authenticated! While bidirectional authentication is certainly supported by SSL, this form of asymmetric authentication is more commonly employed today since most users don’t have certificates.

    Microsoft’s Server Gated Cryptography (SGC) protocol is another extension to SSL/TLS. For several decades, it has been illegal to generally export products from the U.S. that employed secret-key cryptography with keys longer than 40 bits. For that reason, SSL/TLS has an exportable version with weak (40-bit) keys and a domestic (North American) version with strong (128-bit) keys. Within the last several years, however, use of strong SKC has been approved for the worldwide financial community. SGC is an extension to SSL that allows financial institutions using Windows NT servers to employ strong cryptography. Both the client and server must implement SGC and the bank must have a valid SGC certificate. During the initial handshake, the server will indicate support of SGC and supply its SGC certificate; if the client wishes to use SGC and validates the server’s SGC certificate, the session can employ 128-bit RC2, 128-bit RC4, 56-bit DES, or 168-bit 3DES. Microsoft supports SGC in the Windows 95/98/NT versions of Internet Explorer 4.0, Internet Information Server (IIS) 4.0, and Money 98.

    As mentioned above, SSL was designed to provide application-independent transaction security for the Internet. Although the discussion above has focused on HTTP over SSL (https/TCP port 443), SSL is also applicable to:

    Protocol TCP Port Name/Number
    File Transfer Protocol (FTP) ftps-data/989 & ftps/990
    Internet Message Access Protocol v4 (IMAP4) imaps/993
    Lightweight Directory Access Protocol (LDAP) ldaps/636
    Network News Transport Protocol (NNTP) nntps/563
    Post Office Protocol v3 (POP3) pop3s/995
    Telnet telnets/992

    5.8. Elliptic Curve Cryptography

    In general, public-key cryptography systems use hard-to-solve problems as the basis of the algorithm. The most predominant algorithm today for public-key cryptography is RSA, based on the prime factors of very large integers. While RSA can be successfully attacked, the mathematics of the algorithm have not been comprised, per se; instead, computational brute-force has broken the keys. The defense is “simple” — keep the size of the integer to be factored ahead of the computational curve!

    In 1985, Elliptic Curve Cryptography (ECC) was proposed independently by cryptographers Victor Miller (IBM) and Neal Koblitz (University of Washington). ECC is based on the difficulty of solving the Elliptic Curve Discrete Logarithm Problem (ECDLP). Like the prime factorization problem, ECDLP is another “hard” problem that is deceptively simple to state: Given two points, P and Q, on an elliptic curve, find the integer n, if it exists, such that P = nQ.

    Elliptic curves combine number theory and algebraic geometry. These curves can be defined over any field of numbers (i.e., real, integer, complex) although we generally see them used over finite fields for applications in cryptography. An elliptic curve consists of the set of real numbers (x,y) that satisfies the equation:

    y2 = x3 + ax + b

    The set of all of the solutions to the equation forms the elliptic curve. Changing a and b changes the shape of the curve, and small changes in these parameters can result in major changes in the set of (x,y) solutions.

    FIGURE 16: Elliptic curve addition.

    Figure 16 shows the addition of two points on an elliptic curve. Elliptic curves have the interesting property that adding two points on the elliptic curve yields a third point on the curve. Therefore, adding two points, P and Q, gets us to point R, also on the curve. Small changes in P or Q can cause a large change in the position of R.

    So let’s go back to the original problem statement from above. The point Q is calculated as a multiple of the starting point, P, or, Q = nP. An attacker might know P and Q but finding the integer, n, is a difficult problem to solve. Q (i.e., nP) is the public key and n is the private key.

    ECC may be employed with many Internet standards, including CCITT X.509 certificates and certificate revocation lists (CRLs), Internet Key Exchange (IKE), Transport Layer Security (TLS), XML signatures, and applications or protocols based on the cryptographic message syntax (CMS). RFC 5639 proposes a set of elliptic curve domain parameters over finite prime fields for use in these cryptographic applications.

    RSA had been the mainstay of PKC for over a quarter-century. ECC, however, is emerging as a replacement in some environments because it provides similar levels of security compared to RSA but with significantly reduced key sizes. NIST use the following table to demonstrate the key size relationship between ECC and RSA, and the appropriate choice of AES key size:

    TABLE 4. ECC and RSA Key Comparison.
    ECC Key Size RSA Key Size Key-Size
    Ratio
    AES Key Size
    163 1,024 1:6 n/a
    256 3,072 1:12 128
    384 7,680 1:20 192
    512 15,360 1:30 256
    Key sizes in bits. Source: Certicom, NIST

    Since the ECC key sizes are so much shorter than comparable RSA keys, the length of the public key and private key is much shorter in elliptic curve cryptosystems. This results into faster processing times, and lower demands on memory and bandwidth; some studies have found that ECC is faster than RSA for signing and decryption, but slower for signature verification and encryption.

    ECC is particularly useful in applications where memory, bandwidth, and/or computational power is limited (e.g., a smartcard) and it is in this area that ECC use is expected to grow. A major champion of ECC today is Certicom; readers are urged to see their ECC tutorial.

    5.9. The Advanced Encryption Standard and Rijndael

    The search for a replacement to DES started in January 1997 when NIST announced that it was looking for an Advanced Encryption Standard. In September of that year, they put out a formal Call for Algorithms and in August 1998 announced that 15 candidate algorithms were being considered (Round 1). In April 1999, NIST announced that the 15 had been whittled down to five finalists (Round 2):MARS (multiplication, addition, rotation and substitution) from IBM; Ronald Rivest’s RC6Rijndael from a Belgian team; Serpent, developed jointly by a team from England, Israel, and Norway; andTwofish, developed by Bruce Schneier. In October 2000, NIST announced their selection: Rijndael.

    The remarkable thing about this entire process has been the openness as well as the international nature of the “competition.” NIST maintained an excellent Web site devoted to keeping the public fully informed, at http://csrc.nist.gov/archive/aes/, which is now available as an archive site. Their Overview of the AES Development Effort has full details of the process, algorithms, and comments so I will not repeat everything here.

    In October 2000, NIST released the Report on the Development of the Advanced Encryption Standard (AES) that compared the five Round 2 algorithms in a number of categories. The table below summarizes the relative scores of the five schemes (1=low, 3=high):

    Algorithm
    Category MARS RC6 Rijndael Serpent Twofish
    General security 3 2 2 3 3
    Implementation of security 1 1 3 3 2
    Software performance 2 2 3 1 1
    Smart card performance 1 1 3 3 2
    Hardware performance 1 2 3 3 2
    Design features 2 1 2 1 3

    With the report came the recommendation that Rijndael be named the AES. In February 2001, NIST released the Draft Federal Information Processing Standard (FIPS) AES Specification for public review and comment. AES contains a subset of Rijndael’s capabilities (e.g., AES only supports a 128-bit block size) and uses some slightly different nomenclature and terminology, but to understand one is to understand both. The 90-day comment period ended on May 29, 2001 and the U.S. Department of Commerce officially adopted AES in December 2001, published as FIPS PUB 197.

    AES (Rijndael) Overview

    Rijndael (pronounced as in “rain doll” or “rhine dahl”) is a block cipher designed by Joan Daemen and Vincent Rijmen, both cryptographers in Belgium. Rijndael can operate over a variable-length block using variable-length keys; the version 2 specification submitted to NIST describes use of a 128-, 192-, or 256-bit key to encrypt data blocks that are 128, 192, or 256 bits long; note that all nine combinations of key length and block length are possible. The algorithm is written in such a way that block length and/or key length can easily be extended in multiples of 32 bits and it is specifically designed for efficient implementation in hardware or software on a range of processors. The design of Rijndael was strongly influenced by the block cipher called Square, also designed by Daemen and Rijmen.

    Rijndael is an iterated block cipher, meaning that the initial input block and cipher key undergoes multiple rounds of transformation before producing the output. Each intermediate cipher result is called aState.

    For ease of description, the block and cipher key are often represented as an array of columns where each array has 4 rows and each column represents a single byte (8 bits). The number of columns in an array representing the state or cipher key, then, can be calculated as the block or key length divided by 32 (32 bits = 4 bytes). An array representing a State will have Nb columns, where Nb values of 4, 6, and 8 correspond to a 128-, 192-, and 256-bit block, respectively. Similarly, an array representing a Cipher Key will have Nk columns, where Nk values of 4, 6, and 8 correspond to a 128-, 192-, and 256-bit key, respectively. An example of a 128-bit State (Nb=4) and 192-bit Cipher Key (Nk=6) is shown below:

    s0,0 s0,1 s0,2 s0,3
    s1,0 s1,1 s1,2 s1,3
    s2,0 s2,1 s2,2 s2,3
    s3,0 s3,1 s3,2 s3,3
    k0,0 k0,1 k0,2 k0,3 k0,4 k0,5
    k1,0 k1,1 k1,2 k1,3 k1,4 k1,5
    k2,0 k2,1 k2,2 k2,3 k2,4 k2,5
    k3,0 k3,1 k3,2 k3,3 k3,4 k3,5

    The number of transformation rounds (Nr) in Rijndael is a function of the block length and key length, and is given by the table below:

    No. of Rounds
    Nr
    Block Size
    128 bits
    Nb = 4
    192 bits
    Nb = 6
    256 bits
    Nb = 8
    Key
    Size
    128 bits
    Nk = 4
    10 12 14
    192 bits
    Nk = 6
    12 12 14
    256 bits
    Nk = 8
    14 14 14

    Now, having said all of this, the AES version of Rijndael does not support all nine combinations of block and key lengths, but only the subset using a 128-bit block size. NIST calls these supported variants AES-128, AES-192, and AES-256 where the number refers to the key size. The NbNk, and Nr values supported in AES are:

    Parameters
    Variant Nb Nk Nr
    AES-128 4 4 10
    AES-192 4 6 12
    AES-256 4 8 14

    The AES/Rijndael cipher itself has three operational stages:

    • AddRound Key transformation
    • Nr-1 Rounds comprising:
      • SubBytes transformation
      • ShiftRows transformation
      • MixColumns transformation
      • AddRoundKey transformation
    • A final Round comprising:
      • SubBytes transformation
      • ShiftRows transformation
      • AddRoundKey transformation

    The paragraphs below will describe the operations mentioned above. The nomenclature used below is taken from the AES specification although references to the Rijndael specification are made for completeness. The arrays s and s’ refer to the State before and after a transformation, respectively (NOTE: The Rijndael specification uses the array nomenclature a and b to refer to the before and after States, respectively). The subscripts i and j are used to indicate byte locations within the State (or Cipher Key) array.

    The SubBytes transformation

     

    The substitute bytes (called ByteSub in Rijndael) transformation operates on each of the State bytes independently and changes the byte value. An S-box, or substitution table, controls the transformation. The characteristics of the S-box transformation as well as a compliant S-box table are provided in the AES specification; as an example, an input State byte value of 107 (0x6b) will be replaced with a 127 (0x7f) in the output State and an input value of 8 (0x08) would be replaced with a 48 (0x30).

     

    One way to think of the SubBytes transformation is that a given byte in State s is given a new value in State s’ according to the S-box. The S-box, then, is a function on a byte in State s so that:

    s’i,j = S-box (si,j)

    The more general depiction of this transformation is shown by:

    s0,0 s0,1 s0,2 s0,3
    s1,0 s1,1 s1,2 s1,3
    s2,0 s2,1 s2,2 s2,3
    s3,0 s3,1 s3,2 s3,3
    ====>
    S-box
    ====>
    s’0,0 s’0,1 s’0,2 s’0,3
    s’1,0 s’1,1 s’1,2 s’1,3
    s’2,0 s’2,1 s’2,2 s’2,3
    s’3,0 s’3,1 s’3,2 s’3,3

    The ShiftRows transformation

     

    The shift rows (called ShiftRow in Rijndael) transformation cyclically shifts the bytes in the bottom three rows of the State array. According to the more general Rijndael specification, rows 2, 3, and 4 are cyclically left-shifted by C1, C2, and C3 bytes, respectively, per the table below:

    Nb C1 C2 C3
    4 1 2 3
    6 1 2 3
    8 1 3 4

    The current version of AES, of course, only allows a block size of 128 bits (Nb = 4) so that C1=1, C2=2, and C3=3. The diagram below shows the effect of the ShiftRows transformation on State s:

    State s
    s0,0 s0,1 s0,2 s0,3
    s1,0 s1,1 s1,2 s1,3
    s2,0 s2,1 s2,2 s2,3
    s3,0 s3,1 s3,2 s3,3
     
    ———– no shift ———–> 
    —-> left-shift by C1 (1) —-> 
    —-> left-shift by C2 (2) —-> 
    —-> left-shift by C3 (3) —-> 
    State s’
    s0,0 s0,1 s0,2 s0,3
    s1,1 s1,2 s1,3 s1,0
    s2,2 s2,3 s2,0 s2,1
    s3,3 s3,0 s3,1 s3,2

    The MixColumns transformation

     

    The mix columns (called MixColumn in Rijndael) transformation uses a mathematical function to transform the values of a given column within a State, acting on the four values at one time as if they represented a four-term polynomial. In essence, if you think of MixColumns as a function, this could be written:

    s’i,c = MixColumns (si,c)

    for 0<=i<=3 for some column, c. The column position doesn’t change, merely the values within the column.

    Round Key generation and the AddRoundKey transformation

     

    The AES Cipher Key can be 128, 192, or 256 bits in length. The Cipher Key is used to derive a different key to be applied to the block during each round of the encryption operation. These keys are called the Round Keys and each will be the same length as the block, i.e., Nb 32-bit words (words will be denoted W).

    The AES specification defines a key schedule by which the original Cipher Key (of length Nk 32-bit words) is used to form an Expanded Key. The Expanded Key size is equal to the block size times the number of encryption rounds plus 1, which will provide Nr+1 different keys. (Note that there are Nr encipherment rounds but Nr+1 AddRoundKey transformations.)

    Consider that AES uses a 128-bit block and either 10, 12, or 14 iterative rounds depending upon key length. With a 128-bit key, for example, we would need 1408 bits of key material (128×11=1408), or an Expanded Key size of 44 32-bit words (44×32=1408). Similarly, a 192-bit key would require 1664 bits of key material (128×13), or 52 32-bit words, while a 256-bit key would require 1920 bits of key material (128×15), or 60 32-bit words. The key expansion mechanism, then, starts with the 128-, 192-, or 256-bit Cipher Key and produces a 1408-, 1664-, or 1920-bit Expanded Key, respectively. The original Cipher Key occupies the first portion of the Expanded Key and is used to produce the remaining new key material.

     

    The result is an Expanded Key that can be thought of and used as 11, 13, or 15 separate keys, each used for one AddRoundKey operation. These, then, are the Round Keys. The diagram below shows an example using a 192-bit Cipher Key (Nk=6), shown in magenta italics:

    Expanded Key: W0 W1 W2 W3 W4 W5 W6 W7 W8 W9 W10 W11 W12 W13 W14 W15 W44 W45 W46 W47 W48 W49 W50 W51
    Round keys: Round key 0 Round key 1 Round key 2 Round key 3 Round key 11 Round key 12

    The AddRoundKey (called Round Key addition in Rijndael) transformation merely applies each Round Key, in turn, to the State by a simple bit-wise exclusive OR operation. Recall that each Round Key is the same length as the block.

    Summary

    Ok, I hope that you’ve enjoyed reading this as much as I’ve enjoyed writing it — and now let me guide you out of the microdetail! Recall from the beginning of the AES overview that the cipher itself comprises a number of rounds of just a few functions:

    • SubBytes takes the value of a word within a State and substitutes it with another value by a predefined S-box
    • ShiftRows circularly shifts each row in the State by some number of predefined bytes
    • MixColumns takes the value of a 4-word column within the State and changes the four values using a predefined mathematical function
    • AddRoundKey XORs a key that is the same length as the block, using an Expanded Key derived from the original Cipher Key
    Cipher (byte in[4*Nb], byte out[4*Nb], word w[Nb*(Nr+1)])
    begin
      byte state[4,Nb]
    
      state = in
    
      AddRoundKey(state, w)
    
      for round = 1 step 1 to Nr-1
        SubBytes(state)
        ShiftRows(state)
        MixColumns(state)
        AddRoundKey(state, w+round*Nb)
      end for
    
      SubBytes(state)
      ShiftRows(state)
      AddRoundKey(state, w+Nr*Nb)
    
      out = state
    end

    FIGURE 17: AES pseudocode.

    As a last and final demonstration of the operation of AES, Figure 17 is a pseudocode listing for the operation of the AES cipher. In the code:

    • in[] and out[] are 16-byte arrays with the plaintext and cipher text, respectively. (According to the specification, both of these arrays are actually 4*Nb bytes in length but Nb=4 in AES.)
    • state[] is a 2-dimensional array containing bytes in 4 rows and 4 columns. (According to the specification, this arrays is 4 rows by Nb columns.)
    • w[] is an array containing the key material and is 4*(Nr+1) words in length. (Again, according to the specification, the multiplier is actually Nb.)
    • AddRoundKey()SubBytes()ShiftRows(), and MixColumns() are functions representing the individual transformations.

    5.10. Cisco’s Stream Cipher

    Stream ciphers take advantage of the fact that:

    x XOR y XOR y = x

    One of the encryption schemes employed by Cisco routers to encrypt passwords is a stream cipher. It uses the following fixed keystream (thanks also to Jason Fossen for independently extending and confirming this string):

    dsfd;kfoA,.iyewrkldJKDHSUBsgvca69834ncx

    When a password is to be encrypted, the password function chooses a number between 0 and 15, and that becomes the offset into the keystream. Password characters are then XORed byte-by-byte with the keystream according to:

    Ci = Pi XOR K(offset+i)

    where K is the keystream, P is the plaintext password, and C is the ciphertext password.

    Consider the following example. Suppose we have the password abcdefgh. Converting the ASCII characters yields the hex string 0x6162636465666768.

    The keystream characters and hex code that supports an offset from 0 to 15 bytes and a password length up to 24 bytes is:

      d s f d ; k f o A , . i y e w r k l d J K D H S U B s g v c a 6 9 8 3 4 n c x
    0x647366643b6b666f412c2e69796577726b6c644a4b4448535542736776636136393833346e6378

    Let’s say that the function decides upon a keystream offset of 6 bytes. We then start with byte 6 of the keystream (start counting the offset at 0) and XOR with the password:

        0x666f412c2e697965
    XOR 0x6162636465666768
    ——————
    0x070D22484B0F1E0D

    The password would now be displayed in the router configuration as:

    password 7 06070D22484B0F1E0D

    where the “7” indicates the encryption type, the leading “06” indicates the offset into the keystream, and the remaining bytes are the encrypted password characters. (Decryption is pretty trivial so that exercise is left to the reader. If you need some help with byte-wise XORing, see http://www.garykessler.net/library/byte_logic_table.html.)

    5.11. TrueCrypt

    TrueCrypt is an open source, on-the-fly crypto system that can be used on devices supports by Linux, MacOS, and Windows. TrueCrypt can be employed to encrypt individual files, a partition on a disk, or an entire disk.

    TrueCrypt uses a variety of encryption schemes, including AES, Serpent, and Twofish. A TrueCrypt volume is stored as a file that appears to be filled with random data, thus has no easily searchable file signature.

    When a user creates a TrueCrypt volume, a number of parameters need to be defined, such as the size of the volume and the password. To access the volume, the TrueCrypt program is employed to find the TrueCrypt encrypted file, which is then mounted as a new drive on the host system.

    FIGURE 18: TrueCrypt screen shot (Windows).

    FIGURE 19: TrueCrypt screen shot (MacOS).

    Consider this example where an encrypted TrueCrypt volume is stored as a file named James on a thumb drive. On a Windows system, this thumb drive has been mounted as device E:. The TrueCrypt application is used to mount this volume; in this case, the user has chosen to mount the TrueCrypt volume as device K: (Figure 18). Alternatively, the thumb drive could be used with a Mac system, where it has been mounted as the /Volumes/JIMMY volume. TrueCrypt mounts the encrypted file, James, and it is now accessible to the system (Figure 19).

    FIGURE 20: TrueCrypt hidden encrypted volume within an encrypted volume
    (from http://www.truecrypt.org/images/docs/hidden-volume.gif).

    One of the controversial features of TrueCrypt is called plausible deniability, protection in case a user is “forced” to turn over the encrypted volume’s password. When the user creates a TrueCrypt volume, he/she chooses whether to create a standard or hidden volume. A standard volume has a single password. A hidden volume is created within a standard volume and uses a second password. As shown in Figure 20, the unallocated (free) space in a TrueCrypt volume is always filled with random data, thus it is impossible to differentiate a hidden encrypted volume from free space.

    To access the hidden volume, the file is mounted as shown above and the user enters the hidden volume’s password. When under duress, the user would merely enter the password of the “non-hidden” TrueCrypt volume.

    Complete information about TrueCrypt can be found at the TrueCrypt Web Site or in the TrueCrypt Tutorial.

    6. CONCLUSION… OF SORTS

    This paper has briefly described how cryptography works. The reader must beware, however, that there are a number of ways to attack every one of these systems; cryptanalysis and attacks on cryptosystems, however, are well beyond the scope of this paper. In the words of Sherlock Holmes (ok, Arthur Conan Doyle, really), “What one man can invent, another can discover” (“The Adventure of the Dancing Men”).

    Cryptography is a particularly interesting field because of the amount of work that is, by necessity, done in secret. The irony is that today, secrecy is not the key to the goodness of a cryptographic algorithm. Regardless of the mathematical theory behind an algorithm, the best algorithms are those that are well-known and well-documented because they are also well-tested and well-studied! In fact,time is the only true test of good cryptography; any cryptographic scheme that stays in use year after year is most likely a good one. The strength of cryptography lies in the choice (and management) of the keys; longer keys will resist attack better than shorter keys.

    The corollary to this is that consumers should run, not walk, away from any product that uses a proprietary cryptography scheme, ostensibly because the algorithm’s secrecy is an advantage. This observation about not using “secret” crypto schemes has been a fundamental hallmark of cryptography for well over 100 years; it was first stated explicitly by Dutch linguist Auguste Kerckhoffs von Nieuwenhoff in his 1883 (yes, 1883) text titled La Cryptographie militaire, and has therefore become known as “Kerckhoffs’ Principle.”

    7. REFERENCES AND FURTHER READING

    And for a purely enjoyable fiction book that combines cryptography and history, check out Neal Stephenson’s Crytonomicon (published May 1999). You will also find in it a new secure crypto scheme based upon an ordinary deck of cards (ok, you need the jokers…) called the Solitaire Encryption Algorithm, developed by Bruce Schneier.

    Finally, I am not in the clothing business although I do have an impressive t-shirt collection (over 350 and counting!). If you want to proudly wear the DES (well, actually the IDEA) encryption algorithm, be sure to see 2600 Magazine‘s DES Encryption Shirt, found athttp://store.yahoo.com/2600hacker/desenshir.html (left). A t-shirt with Adam Back’s RSA Perl code can be found athttp://www.cypherspace.org/~adam/uk-shirt.html (right).

    APPENDIX. SOME MATH NOTES

    A number of readers over time have asked for some rudimentary background on a few of the less well-known mathematical functions mentioned in this paper. Although this is purposely not a mathematical treatise, some of the math functions mentioned here are essential to grasping how modern crypto functions work. To that end, some of the mathematical functions mentioned in this paper are defined in greater detail below.

    A.1. The Exclusive-OR (XOR) Function

    Exclusive OR (XOR) is one of the fundamental mathematical operations used in cryptography (and many other applications). George Boole, a mathematician in the late 1800s, invented a new form of “algebra” that provides the basis for building electronic computers and microprocessor chips. Boole defined a bunch of primitive logical operations where there are one or two inputs and a single output depending upon the operation; the input and output are either TRUE or FALSE. The most elemental Boolean operations are:

    • NOT: The output value is the inverse of the input value (i.e., the output is TRUE if the input is false, FALSE if the input is true)
    • AND: The output is TRUE if all inputs are true, otherwise FALSE. (E.g., “the sky is blue AND the world is flat” is FALSE while “the sky is blue AND security is a process” is TRUE.)
    • OR: The output is TRUE if either or both inputs are true, otherwise FALSE. (E.g., “the sky is blue OR the world is flat” is TRUE and “the sky is blue OR security is a process” is TRUE.)
    • XOR (Exclusive OR): The output is TRUE if exactly one of the inputs is TRUE, otherwise FALSE. (E.g., “the sky is blue XOR the world is flat” is TRUE while “the sky is blue XOR security is a process” is FALSE.)

    I’ll only discuss XOR for now and demonstrate its function by the use of a so-called truth tables. In computers, Boolean logic is implemented in logic gates; for design purposes, XOR has two inputs and a single output, and its logic diagram looks like this:

    XOR 0 1
    0 0 1
    1 1 0

    So, in an XOR operation, the output will be a 1 if one input is a 1; otherwise, the output is 0. The real significance of this is to look at the “identity properties” of XOR. In particular, any value XORed with itself is 0 and any value XORed with 0 is just itself. Why does this matter? Well, if I take my plaintext and XOR it with a key, I get a jumble of bits. If I then take that jumble and XOR it with the same key, I return to the original plaintext.

    NOTE: Boolean truth tables usually show the inputs and output as a single bit because they are based on single bit inputs, namely, TRUE and FALSE. In addition, we tend to apply Boolean operations bit-by-bit. For convenience, I have created Boolean logic tables when operating on bytes.

    A.2. The modulo Function

    The modulo function is, simply, the remainder function. It is commonly used in programming and is critical to the operation of any mathematical function using digital computers.

    To calculate X modulo Y (usually written X mod Y), you merely determine the remainder after removing all multiples of Y from X. Clearly, the value X mod Y will be in the range from 0 to Y-1.

    Some examples should clear up any remaining confusion:

    • 15 mod 7 = 1
    • 25 mod 5 = 0
    • 33 mod 12 = 9
    • 203 mod 256 = 203

    Modulo arithmetic is useful in crypto because it allows us to set the size of an operation and be sure that we will never get numbers that are too large. This is an important consideration when using digital computers.

    ABOUT THE AUTHOR

    Gary C. Kessler, Ph.D., CCE, CISSP, is the president and janitor of Gary Kessler Associates, an independent consulting and training firm specializing in computer and network security, computer forensics, Internet access issues, and TCP/IP networking. He has written over 60 papers for industry publications, is co-author of ISDN, 4th. edition (McGraw-Hill, 1998), and is on the editorial boards of the Journal of Digital Forensic Practice and the Journal of Digital Forensics, Security and Law. Gary is also a member of the Vermont Internet Crimes Against Children (ICAC) Task Force and an adjunct faculty member at Edith Cowan University in Perth, Western Australia. Gary is also an adjunct faculty member at Champlain College in Burlington, VT, where he started the M.S. in Digital Investigation Management and undergraduate Computer & Digital Forensics programs. Gary’s e-mail address is kumquat@sover.net and his PGP public key can be found athttp://www.garykessler.net/kumquat_pubkey.html or on MIT’s PGP keyserver. Some of Gary’s other crypto pointers of interest on the Web can be found at his Security-related URLs list.

Hacker

The Link to this document is http://en.wikipedia.org/wiki/Hack_(computer_security)

Hacker (computer security)

From Wikipedia, the free encyclopedia
  (Redirected from Hack (computer security))
Page move-protected
This article is part of a series on
Computer security hacking
History
PhreakingCryptovirology
Hacker ethic
Hacker ManifestoBlack hatGrey hat,White hatBlack Hat BriefingsDEF CON
Cybercrime
Computer crimeCrimeware,List of convicted computer criminals,Script kiddie
Hacking tools
VulnerabilityExploitPayload
Malware
RootkitBackdoorTrojan horseVirus,WormSpywareBotnet,Keystroke loggingAntivirus software,FirewallHIDS
Computer security
Computer insecurityApplication security,Network security
v · d · e

hacker is a person who breaks into computers and computer networks for profit, as protest, or sometimes by the motivation of the challenge.[1] The subculture that has evolved around hackers is often referred to as the computer underground but is now an open community.[2]

Other definitions of the word hacker exist that are not related to computer security. They are subject to the long standing hacker definition controversy about the true meaning of hacker. In this controversy, the term hacker is reclaimed by computer programmers who argue that someone breaking into computers is better called cracker,[3] not making a difference between computer criminals (“black hats”) and computer security experts (“white hats”). Some white hat hackers claim that they also deserve the title hacker, and that only black hats should be called crackers.

Contents

[hide]

[edit]History

In today’s society understanding the term “hacker” is complicated because it has many different definitions. The term can be traced back to MIT (Massachusetts Institute Technology). MIT was the first institution to offer a course in computer programming and computer science and it is here in 1960 where a group of MIT students taking a lab on artificial intelligence first coined this word. These students called themselves hackers because they were able to take programs and have them perform actions not intended for that program. “The term was developed on the basis of a practical joke and feeling of excitement because the team member would “hack away” at the keyboard hours at a time.” (Moore R., 2006).[4]

Hacking developed alongside phone phreaking, a term referred to exploration of the phone network without authorization, and there has often been overlap between both technology and participants.[citation needed] One of the first hacks was accomplished by Joe Engressia also known as The Whistler. Engressia is known as the grandfather of phreaking. His hacking technique was that he could perfectly whistle a tone into a phone and make a free call.[5] Bruce Sterling traces part of the roots of the computer underground to the Yippies, a 1960s counterculture movement which published the Technological Assistance Program (TAP) newsletter.[6] Other sources of early 1970s hacker culture can be traced towards more beneficial forms of hacking, including MIT labs or theHomebrew Computer Club, which later resulted in such things as early personal computers or the open source movement.

[edit]Artifacts and customs

The computer underground[1] is heavily dependent technology. It has produced its own slang and various forms of unusual alphabet use, for example 1337speak. Writing programs and performing other activities to support these views is referred to as hacktivism. Some go as far as seeing illegal cracking ethically justified for this goal; a common form is website defacement. The computer underground is frequently compared to the Wild West.[7] It is common among hackers to use aliases for the purpose of concealing identity, rather than revealing their real names.

[edit]Hacker groups and conventions

Main articles: Hacker conference and Hacker group

The computer underground is supported by regular real-world gatherings called hacker conventions or “hacker cons”. These draw many people every year including SummerCon (Summer), DEF CON,HoHoCon (Christmas), ShmooCon (February), BlackHat, Hacker Halted, and H.O.P.E..[citation needed] In the early 1980s Hacker Groups became popular, Hacker groups provided access to information and resources, and a place to learn from other members. Hackers could also gain credibility by being affiliated with an elite group.[8]

[edit]Hacker attitudes

Several subgroups of the computer underground with different attitudes and aims use different terms to demarcate themselves from each other, or try to exclude some specific group with which they do not agree. Eric S. Raymond (author of The New Hacker’s Dictionary) advocates that members of the computer underground should be called crackers. Yet, those people see themselves as hackers and even try to include the views of Raymond in what they see as one wider hacker culture, a view harshly rejected by Raymond himself. Instead of a hacker/cracker dichotomy, they give more emphasis to a spectrum of different categories, such as white hatgrey hatblack hat and script kiddie. In contrast to Raymond, they usually reserve the term cracker. According to (Clifford R.D. 2006) a cracker or cracking is to “gain unauthorized access to a computer in order to commit another crime such as destroying information contained in that system”.[9] These subgroups may also be defined by the legal status of their activities.[10]

White hat
white hat hacker breaks security for non-malicious reasons, for instance testing their own security system. This classification also includes individuals who perform penetration tests andvulnerability assessments within a contractual agreement. Often, this type of ‘white hat’ hacker is called an ethical hacker. The International Council of Electronic Commerce Consultants, also known as the EC-Council has developed certifications, courseware, classes, and online training covering the diverse arena of Ethical Hacking.[10]
Black hat
A Black Hat Hacker is a hacker who “violates computer security for little reason beyond maliciousness or for personal gain”(Moore,2005). Black Hat Hackers are “the epitome of all that the public fears in a computer criminal”(Moore,2006). Black Hat Hackers break into secure networks to destroy data or make the network unusable for those who are authorized to use the network.

The way Black Hat Hackers choose the networks that they are going to break into is by a process that can be broken down into two parts. This is called the pre-hacking stage.

Part 1 Targeting Targeting is when the hacker determines what network to break into. The target may be of particular interest to the hacker, or the hacker may “Port Scan” a network to determine if it is vulnerable to attacks. A port is defined as “an opening through which the computer receives data via the network”(Moore,2005). Open ports will allow a hacker to access the system.

Part 2 Research and Information Gathering It is in this stage that the hacker will visit or contact the target in some way in hopes of finding out vital information that will help them access the system. The main way that hackers get desired results from this stage is from Social Engineering, which will be explained below. Aside from Social Engineering hackers can also use a technique called Dumpster Diving. Dumpster Diving is when a hacker will literally dive into a dumpster in hopes to find documents that users have thrown away, which will help them gain access to a network.

Grey hat
grey hat hacker is a combination of a Black Hat and a White Hat Hacker. A Grey Hat Hacker may surf the internet and hack into a computer system for the sole purpose of notifying the administrator that their system has been hacked, for example. Then they may offer to repair their system for a small fee.[4]
Elite hacker
social status among hackers, elite is used to describe the most skilled. Newly discovered exploits will circulate among these hackers. Elite groups such as Masters of Deception conferred a kind of credibility on their members.[11]:86,90,117 Elite (e.g. 31337) gives the term leet speak its name.
Script kiddie
script kiddie is a non-expert who breaks into computer systems by using pre-packaged automated tools written by others, usually with little understanding of the underlying concept—hence the term script (i.e. a prearranged plan or set of activities) kiddie (i.e. kid, child—an individual lacking knowledge and experience, immature).[12]
Neophyte
A neophyte, “n00b”, or “newbie” is someone who is new to hacking or phreaking and has almost no knowledge or experience of the workings of technology, and hacking.[4]
Blue hat
blue hat hacker is someone outside computer security consulting firms who is used to bug test a system prior to its launch, looking for exploits so they can be closed. Microsoft also uses the term BlueHat to represent a series of security briefing events.[13][14][15]
Hacktivist
A hacktivist is a hacker who utilizes technology to announce a social, ideological, religious, or political message. In general, most hacktivism involves website defacement or denial-of-service attacks. In more extreme cases, hacktivism is used as tool for cyberterrorism.

[edit]Attacks

Main article: Computer insecurity
Computer security
Secure operating systems
Security architecture
Security by design
Secure coding
Computer insecurity
Vulnerability Social engineering
Eavesdropping
Exploits Trojans
Viruses and worms
Denial of service
Payloads Backdoors
Rootkits
Keyloggers
v · d · e

A typical approach in an attack on Internet-connected system is:

  1. Network enumeration: Discovering information about the intended target.
  2. Vulnerability analysis: Identifying potential ways of attack.
  3. Exploitation: Attempting to compromise the system by employing the vulnerabilities found through the vulnerability analysis.[16]

In order to do so, there are several recurring tools of the trade and techniques used by computer criminals and security experts.

[edit]Security exploits

A security exploit is a prepared application that takes advantage of a known weakness. Common examples of security exploits are SQL injectionCross Site Scripting and Cross Site Request Forgery which abuse security holes that may result from substandard programming practice. Other exploits would be able to be used through FTPHTTPPHPSSHTelnet and some web-pages. These are very common in website/domain hacking.

[edit]Techniques

Vulnerability scanner
vulnerability scanner is a tool used to quickly check computers on a network for known weaknesses. Hackers also commonly use port scanners. These check to see which ports on a specified computer are “open” or available to access the computer, and sometimes will detect what program or service is listening on that port, and its version number. (Note that firewalls defend computers from intruders by limiting access to ports/machines both inbound and outbound, but can still be circumvented.)
Password cracking
Password cracking is the process of recovering passwords from data that has been stored in or transmitted by a computer system. A common approach is to repeatedly try guesses for the password.
Packet sniffer
packet sniffer is an application that captures data packets, which can be used to capture passwords and other data in transit over the network.
Spoofing attack (Phishing)
spoofing attack involves one program, system, or website successfully masquerading as another by falsifying data and thereby being treated as a trusted system by a user or another program. The purpose of this is usually to fool programs, systems, or users into revealing confidential information, such as user names and passwords, to the attacker.
Rootkit
rootkit is designed to conceal the compromise of a computer’s security, and can represent any of a set of programs which work to subvert control of an operating system from its legitimate operators. Usually, a rootkit will obscure its installation and attempt to prevent its removal through a subversion of standard system security. Rootkits may include replacements for system binaries so that it becomes impossible for the legitimate user to detect the presence of the intruder on the system by looking at process tables.
Social engineering
Social engineering When a Hacker, typically a black hat, is in the second stage of the targeting process, he or she will typically use some social engineering tactics to get enough information to access the network. A common practice for hackers who use this technique, is to contact the system administrator and play the role of a user who cannot get access to his or her system. Hackers who use this technique have to be quite savvy and choose the words they use carefully, in order to trick the system administrator into giving them information. In some cases only an employed help desk user will answer the phone and they are generally easy to trick. Another typical hacker approach is for the hacker to act like a very angry supervisor and when the his/her authority is questioned they will threaten the help desk user with their job. Social Engineering is so effective because users are the most vulnerable part of an organization. All the security devices and programs in the world wont keep an organization safe if an employee gives away a password. Black Hat Hackers take advantage of this fact. Social Engineering can also be broken down into four sub-groups. These are intimidation, helpfulness, technical, and name-dropping.

Intimidation As stated above, with the angry supervisor, the hacker attacks the person who answers the phone with threats to their job. Many people at this point will accept that the hacker is a supervisor and give them the needed information.

Helpfulness Opposite to intimidation, helpfulness is taking advantage of a person natural instinct to help someone with a problem. The hacker will not get angry instead act very distressed and concerned. The help desk is the most vulnerable to this type of Social Engineering, because they generally have the authority to change or reset passwords which is exactly what the hacker needs.

Name-Dropping Simply put the hacker uses the names of advanced users as “key words”, and gets the person who answers the phone to believe that they are part of the company because of this. Some information, like web page ownership, can be obtained easily on the web. Other information such as president and vice president names might have to be obtained via dumpster diving.

Technical Using technology to get information is also a great way to get it. A hacker can send a fax or an email to a legitimate user in hopes to get a response containing vital information. Many times the hacker will act like he/she is involved with law enforcement and needs certain data for record keeping purposes or investigations.

Trojan horses
Trojan horse is a program which seems to be doing one thing, but is actually doing another. A trojan horse can be used to set up a back door in a computer system such that the intruder can gain access later. (The name refers to the horse from the Trojan War, with conceptually similar function of deceiving defenders into bringing an intruder inside.)
Viruses
virus is a self-replicating program that spreads by inserting copies of itself into other executable code or documents. Therefore, a computer virus behaves in a way similar to a biological virus, which spreads by inserting itself into living cells.
While some are harmless or mere hoaxes most computer viruses are considered malicious.
Worms
Like a virus, a worm is also a self-replicating program. A worm differs from a virus in that it propagates through computer networks without user intervention. Unlike a virus, it does not need to attach itself to an existing program. Many people conflate the terms “virus” and “worm”, using them both to describe any self-propagating program.
Key loggers
key logger is a tool designed to record (‘log’) every keystroke on an affected machine for later retrieval. Its purpose is usually to allow the user of this tool to gain access to confidential information typed on the affected machine, such as a user’s password or other private data. Some key loggers uses virus-, trojan-, and rootkit-like methods to remain active and hidden. However, some key loggers are used in legitimate ways and sometimes to even enhance computer security. As an example, a business might have a key logger on a computer used at a point of sale and data collected by the key logger could be used for catching employee fraud.

[edit]Training

This is why each organization needs to train their employees with what they should and should not do, security wise. “Training should be creative, varied, related to real life, and frequent.”(Tipton,2007) In addition “The attitude of the trainers should be to raise the awareness and behavior of the attendees to a higher level, not to explain the rules as if to criminals that they had “better behave or else.””(Tipton,2007)”Training for a user must include the proper use of the system and the reasons for the various controls and security parameters built into the system. Without divulging the details of the controls, explaining the reasons for the controls may help the users to accept and adhere to the security restrictions built into the system.”(Tipton,2007) When users are fully trained with how to keep their information and others information secure, they must also be taught to always follow procedure. If an employee is told not to give away a user name or password unless some other important information is provided, then they must not give away that information unless that are 100% sure the user is valid.

Other Useful Tips Other things you can do to prevent Black Hat Hackers from accessing your systems include: Job Rotation and Segregation of Duties. Job Rotation will keep users learning and adapting to different parts of the organization. Job Rotation will also keep users from “getting used to” their job. If a user sits around doing the same thing day after day they tend to slack off more, a business owner will get more “bang for their buck” and overall security using this technique. Segregation of Duties is also vital to network security. Do not give a single user more access to information or more responsibility then is absolutely necessary. A good example of this is having one user input data, and another user to process the data. Should either users information get out, it is much less of a security risk then if they had control over the entire process.

 

[edit]Notable intruders and criminal hackers

[edit]Notable security hackers

Main article: List of hackers

[edit]Hacking and the media

This section is in a list format that may be better presented using prose. You can help by converting this section to prose, if appropriate.Editing help is available. (August 2008)

[edit]Hacker magazines

Main category: Hacker magazines.

The most notable hacker-oriented magazine publications are PhrackHakin9 and 2600: The Hacker Quarterly. While the information contained in hacker magazines and ezines was often outdated, they improved the reputations of those who contributed by documenting their successes.[8]

[edit]Hackers in fiction

Hackers often show an interest in fictional cyberpunk and cyberculture literature and movies. Absorption of fictional pseudonyms, symbols, values, and metaphors from these fictional works is very common.[citation needed]

Books portraying hackers:

Films also portray hackers:

[edit]Non-fiction books

[edit]Fiction books

[edit]See also

[edit]

 

 

 

 

 

 

 

How To Become A Hacker

How To Become A Hacker

Eric Steven Raymond

 

The original document is at http://www.catb.org/~esr/faqs/hacker-howto.html

Thyrsus Enterprises

    <esr@thyrsus.com>
   Â

Copyright © 2001 Eric S. Raymond

Revision History
Revision 1.43 07 Feb 2011 esr
Python passed Perl in popularity in 2010.
Revision 1.42 22 Oct 2010 esr
Added “Historical note”.
Revision 1.40 3 Nov 2008 esr
Link fixes.
Revision 1.39 14 Aug Jan 2008 esr
Link fixes.
Revision 1.38 8 Jan 2008 esr
Deprecate Java as a language to learn early.
Revision 1.37 4 Oct 2007 esr
Recommend Ubuntu as a Unix distro for newbies.

Table of Contents

Why This Document?
What Is a Hacker?
The Hacker Attitude
1. The world is full of fascinating problems waiting to be solved.
2. No problem should ever have to be solved twice.
3. Boredom and drudgery are evil.
4. Freedom is good.
5. Attitude is no substitute for competence.
Basic Hacking Skills
1. Learn how to program.
2. Get one of the open-source Unixes and learn to use and run it.
3. Learn how to use the World Wide Web and write HTML.
4. If you don’t have functional English, learn it.
Status in the Hacker Culture
1. Write open-source software
2. Help test and debug open-source software
3. Publish useful information
4. Help keep the infrastructure working
5. Serve the hacker culture itself
The Hacker/Nerd Connection
Points For Style
Historical Note: Hacking, Open Source, and Free Software
Other Resources
Frequently Asked Questions

Why This Document?

As editor of the Jargon File and author of a few other well-known documents of similar nature, I often get email requests from enthusiastic network newbies asking (in effect) “how can I learn to be a wizardly hacker?”. Back in 1996 I noticed that there didn’t seem to be any other FAQs or web documents that addressed this vital question, so I started this one. A lot of hackers now consider it definitive, and I suppose that means it is. Still, I don’t claim to be the exclusive authority on this topic; if you don’t like what you read here, write your own.

If you are reading a snapshot of this document offline, the current version lives at http://catb.org/~esr/faqs/hacker-howto.html.

Note: there is a list of Frequently Asked Questions at the end of this document. Please read these—twice—before mailing me any questions about this document.

Numerous translations of this document are available: Arabic Belorussian Chinese (Simplified)DanishDutchEstonianGermanGreek Italian HebrewNorwegianPortuguese (Brazilian)Romanian SpanishTurkish, andSwedish. Note that since this document changes occasionally, they may be out of date to varying degrees.

The five-dots-in-nine-squares diagram that decorates this document is called a glider. It is a simple pattern with some surprising properties in a mathematical simulation called Life that has fascinated hackers for many years. I think it makes a good visual emblem for what hackers are like — abstract, at first a bit mysterious-seeming, but a gateway to a whole world with an intricate logic of its own. Read more about the glider emblem here.

What Is a Hacker?

The Jargon File contains a bunch of definitions of the term ‘hacker’, most having to do with technical adeptness and a delight in solving problems and overcoming limits. If you want to know how to become a hacker, though, only two are really relevant.

There is a community, a shared culture, of expert programmers and networking wizards that traces its history back through decades to the first time-sharing minicomputers and the earliest ARPAnet experiments. The members of this culture originated the term ‘hacker’. Hackers built the Internet. Hackers made the Unix operating system what it is today. Hackers run Usenet. Hackers make the World Wide Web work. If you are part of this culture, if you have contributed to it and other people in it know who you are and call you a hacker, you’re a hacker.

The hacker mind-set is not confined to this software-hacker culture. There are people who apply the hacker attitude to other things, like electronics or music — actually, you can find it at the highest levels of any science or art. Software hackers recognize these kindred spirits elsewhere and may call them ‘hackers’ too — and some claim that the hacker nature is really independent of the particular medium the hacker works in. But in the rest of this document we will focus on the skills and attitudes of software hackers, and the traditions of the shared culture that originated the term ‘hacker’.

There is another group of people who loudly call themselves hackers, but aren’t. These are people (mainly adolescent males) who get a kick out of breaking into computers and phreaking the phone system. Real hackers call these people ‘crackers’ and want nothing to do with them. Real hackers mostly think crackers are lazy, irresponsible, and not very bright, and object that being able to break security doesn’t make you a hacker any more than being able to hotwire cars makes you an automotive engineer. Unfortunately, many journalists and writers have been fooled into using the word ‘hacker’ to describe crackers; this irritates real hackers no end.

The basic difference is this: hackers build things, crackers break them.

If you want to be a hacker, keep reading. If you want to be a cracker, go read the alt.2600 newsgroup and get ready to do five to ten in the slammer after finding out you aren’t as smart as you think you are. And that’s all I’m going to say about crackers.

The Hacker Attitude

Hackers solve problems and build things, and they believe in freedom and voluntary mutual help. To be accepted as a hacker, you have to behave as though you have this kind of attitude yourself. And to behave as though you have the attitude, you have to really believe the attitude.

But if you think of cultivating hacker attitudes as just a way to gain acceptance in the culture, you’ll miss the point. Becoming the kind of person who believes these things is important for you â€” for helping you learn and keeping you motivated. As with all creative arts, the most effective way to become a master is to imitate the mind-set of masters — not just intellectually but emotionally as well.

Or, as the following modern Zen poem has it:

    To follow the path:
    look to the master,
    follow the master,
    walk with the master,
    see through the master,
    become the master.

So, if you want to be a hacker, repeat the following things until you believe them:

1. The world is full of fascinating problems waiting to be solved.

Being a hacker is lots of fun, but it’s a kind of fun that takes lots of effort. The effort takes motivation. Successful athletes get their motivation from a kind of physical delight in making their bodies perform, in pushing themselves past their own physical limits. Similarly, to be a hacker you have to get a basic thrill from solving problems, sharpening your skills, and exercising your intelligence.

If you aren’t the kind of person that feels this way naturally, you’ll need to become one in order to make it as a hacker. Otherwise you’ll find your hacking energy is sapped by distractions like sex, money, and social approval.

(You also have to develop a kind of faith in your own learning capacity — a belief that even though you may not know all of what you need to solve a problem, if you tackle just a piece of it and learn from that, you’ll learn enough to solve the next piece — and so on, until you’re done.)

2. No problem should ever have to be solved twice.

Creative brains are a valuable, limited resource. They shouldn’t be wasted on re-inventing the wheel when there are so many fascinating new problems waiting out there.

To behave like a hacker, you have to believe that the thinking time of other hackers is precious — so much so that it’s almost a moral duty for you to share information, solve problems and then give the solutions away just so other hackers can solve new problems instead of having to perpetually re-address old ones.

Note, however, that “No problem should ever have to be solved twice.” does not imply that you have to consider all existing solutions sacred, or that there is only one right solution to any given problem. Often, we learn a lot about the problem that we didn’t know before by studying the first cut at a solution. It’s OK, and often necessary, to decide that we can do better. What’s not OK is artificial technical, legal, or institutional barriers (like closed-source code) that prevent a good solution from being re-used and force people to re-invent wheels.

(You don’t have to believe that you’re obligated to give all your creative product away, though the hackers that do are the ones that get most respect from other hackers. It’s consistent with hacker values to sell enough of it to keep you in food and rent and computers. It’s fine to use your hacking skills to support a family or even get rich, as long as you don’t forget your loyalty to your art and your fellow hackers while doing it.)

3. Boredom and drudgery are evil.

Hackers (and creative people in general) should never be bored or have to drudge at stupid repetitive work, because when this happens it means they aren’t doing what only they can do — solve new problems. This wastefulness hurts everybody. Therefore boredom and drudgery are not just unpleasant but actually evil.

To behave like a hacker, you have to believe this enough to want to automate away the boring bits as much as possible, not just for yourself but for everybody else (especially other hackers).

(There is one apparent exception to this. Hackers will sometimes do things that may seem repetitive or boring to an observer as a mind-clearing exercise, or in order to acquire a skill or have some particular kind of experience you can’t have otherwise. But this is by choice — nobody who can think should ever be forced into a situation that bores them.)

4. Freedom is good.

Hackers are naturally anti-authoritarian. Anyone who can give you orders can stop you from solving whatever problem you’re being fascinated by — and, given the way authoritarian minds work, will generally find some appallingly stupid reason to do so. So the authoritarian attitude has to be fought wherever you find it, lest it smother you and other hackers.

(This isn’t the same as fighting all authority. Children need to be guided and criminals restrained. A hacker may agree to accept some kinds of authority in order to get something he wants more than the time he spends following orders. But that’s a limited, conscious bargain; the kind of personal surrender authoritarians want is not on offer.)

Authoritarians thrive on censorship and secrecy. And they distrust voluntary cooperation and information-sharing — they only like ‘cooperation’ that they control. So to behave like a hacker, you have to develop an instinctive hostility to censorship, secrecy, and the use of force or deception to compel responsible adults. And you have to be willing to act on that belief.

5. Attitude is no substitute for competence.

To be a hacker, you have to develop some of these attitudes. But copping an attitude alone won’t make you a hacker, any more than it will make you a champion athlete or a rock star. Becoming a hacker will take intelligence, practice, dedication, and hard work.

Therefore, you have to learn to distrust attitude and respect competence of every kind. Hackers won’t let posers waste their time, but they worship competence — especially competence at hacking, but competence at anything is valued. Competence at demanding skills that few can master is especially good, and competence at demanding skills that involve mental acuteness, craft, and concentration is best.

If you revere competence, you’ll enjoy developing it in yourself — the hard work and dedication will become a kind of intense play rather than drudgery. That attitude is vital to becoming a hacker.

Basic Hacking Skills

The hacker attitude is vital, but skills are even more vital. Attitude is no substitute for competence, and there’s a certain basic toolkit of skills which you have to have before any hacker will dream of calling you one.

This toolkit changes slowly over time as technology creates new skills and makes old ones obsolete. For example, it used to include programming in machine language, and didn’t until recently involve HTML. But right now it pretty clearly includes the following:

1. Learn how to program.

This, of course, is the fundamental hacking skill. If you don’t know any computer languages, I recommend starting with Python. It is cleanly designed, well documented, and relatively kind to beginners. Despite being a good first language, it is not just a toy; it is very powerful and flexible and well suited for large projects. I have written a more detailed evaluation of Python. Good tutorials are available at the Python web site.

I used to recommend Java as a good language to learn early, but this critique has changed my mind (search for â€œThe Pitfalls of Java as a First Programming Language” within it). A hacker cannot, as they devastatingly put it â€œapproach problem-solving like a plumber in a hardware store”; you have to know what the components actually do. Now I think it is probably best to learn C and Lisp first, then Java.

There is perhaps a more general point here. If a language does too much for you, it may be simultaneously a good tool for production and a bad one for learning. It’s not only languages that have this problem; web application frameworks like RubyOnRails, CakePHP, Django may make it too easy to reach a superficial sort of understanding that will leave you without resources when you have to tackle a hard problem, or even just debug the solution to an easy one.

If you get into serious programming, you will have to learn C, the core language of Unix. C++ is very closely related to C; if you know one, learning the other will not be difficult. Neither language is a good one to try learning as your first, however. And, actually, the more you can avoid programming in C the more productive you will be.

C is very efficient, and very sparing of your machine’s resources. Unfortunately, C gets that efficiency by requiring you to do a lot of low-level management of resources (like memory) by hand. All that low-level code is complex and bug-prone, and will soak up huge amounts of your time on debugging. With today’s machines as powerful as they are, this is usually a bad tradeoff — it’s smarter to use a language that uses the machine’s time less efficiently, but your time much more efficiently. Thus, Python.

Other languages of particular importance to hackers include Perl and LISP. Perl is worth learning for practical reasons; it’s very widely used for active web pages and system administration, so that even if you never write Perl you should learn to read it. Many people use Perl in the way I suggest you should use Python, to avoid C programming on jobs that don’t require C’s machine efficiency. You will need to be able to understand their code.

LISP is worth learning for a different reason — the profound enlightenment experience you will have when you finally get it. That experience will make you a better programmer for the rest of your days, even if you never actually use LISP itself a lot. (You can get some beginning experience with LISP fairly easily by writing and modifying editing modes for the Emacs text editor, or Script-Fu plugins for the GIMP.)

It’s best, actually, to learn all five of Python, C/C++, Java, Perl, and LISP. Besides being the most important hacking languages, they represent very different approaches to programming, and each will educate you in valuable ways.

But be aware that you won’t reach the skill level of a hacker or even merely a programmer simply by accumulating languages — you need to learn how to think about programming problems in a general way, independent of any one language. To be a real hacker, you need to get to the point where you can learn a new language in days by relating what’s in the manual to what you already know. This means you should learn several very different languages.

I can’t give complete instructions on how to learn to program here — it’s a complex skill. But I can tell you that books and courses won’t do it — many, maybe most of the best hackers are self-taught. You can learn language features — bits of knowledge — from books, but the mind-set that makes that knowledge into living skill can be learned only by practice and apprenticeship. What will do it is (a) reading code and (b) writing code.

Peter Norvig, who is one of Google’s top hackers and the co-author of the most widely used textbook on AI, has written an excellent essay called Teach Yourself Programming in Ten Years. His “recipe for programming success” is worth careful attention.

Learning to program is like learning to write good natural language. The best way to do it is to read some stuff written by masters of the form, write some things yourself, read a lot more, write a little more, read a lot more, write some more … and repeat until your writing begins to develop the kind of strength and economy you see in your models.

Finding good code to read used to be hard, because there were few large programs available in source for fledgeling hackers to read and tinker with. This has changed dramatically; open-source software, programming tools, and operating systems (all built by hackers) are now widely available. Which brings me neatly to our next topic…

2. Get one of the open-source Unixes and learn to use and run it.

I’ll assume you have a personal computer or can get access to one. (Take a moment to appreciate how much that means. The hacker culture originally evolved back when computers were so expensive that individuals could not own them.) The single most important step any newbie can take toward acquiring hacker skills is to get a copy of Linux or one of the BSD-Unixes or OpenSolaris, install it on a personal machine, and run it.

Yes, there are other operating systems in the world besides Unix. But they’re distributed in binary — you can’t read the code, and you can’t modify it. Trying to learn to hack on a Microsoft Windows machine or under any other closed-source system is like trying to learn to dance while wearing a body cast.

Under Mac OS X it’s possible, but only part of the system is open source — you’re likely to hit a lot of walls, and you have to be careful not to develop the bad habit of depending on Apple’s proprietary code. If you concentrate on the Unix under the hood you can learn some useful things.

Unix is the operating system of the Internet. While you can learn to use the Internet without knowing Unix, you can’t be an Internet hacker without understanding Unix. For this reason, the hacker culture today is pretty strongly Unix-centered. (This wasn’t always true, and some old-time hackers still aren’t happy about it, but the symbiosis between Unix and the Internet has become strong enough that even Microsoft’s muscle doesn’t seem able to seriously dent it.)

So, bring up a Unix — I like Linux myself but there are other ways (and yes, you can run both Linux and Microsoft Windows on the same machine). Learn it. Run it. Tinker with it. Talk to the Internet with it. Read the code. Modify the code. You’ll get better programming tools (including C, LISP, Python, and Perl) than any Microsoft operating system can dream of hosting, you’ll have fun, and you’ll soak up more knowledge than you realize you’re learning until you look back on it as a master hacker.

For more about learning Unix, see The Loginataka. You might also want to have a look at The Art Of Unix Programming.

To get your hands on a Linux, see the Linux Online! site; you can download from there or (better idea) find a local Linux user group to help you with installation.

During the first ten years of this HOWTO’s life, I reported that from a new user’s point of view, all Linux distributions are almost equivalent. But in 2006-2007, an actual best choice emerged: Ubuntu. While other distros have their own areas of strength, Ubuntu is far and away the most accessible to Linux newbies.

You can find BSD Unix help and resources at www.bsd.org.

A good way to dip your toes in the water is to boot up what Linux fans call a live CD, a distribution that runs entirely off a CD without having to modify your hard disk. This will be slow, because CDs are slow, but it’s a way to get a look at the possibilities without having to do anything drastic.

I have written a primer on the basics of Unix and the Internet.

I used to recommend against installing either Linux or BSD as a solo project if you’re a newbie. Nowadays the installers have gotten good enough that doing it entirely on your own is possible, even for a newbie. Nevertheless, I still recommend making contact with your local Linux user’s group and asking for help. It can’t hurt, and may smooth the process.

3. Learn how to use the World Wide Web and write HTML.

Most of the things the hacker culture has built do their work out of sight, helping run factories and offices and universities without any obvious impact on how non-hackers live. The Web is the one big exception, the huge shiny hacker toy that even politicians admit has changed the world. For this reason alone (and a lot of other good ones as well) you need to learn how to work the Web.

This doesn’t just mean learning how to drive a browser (anyone can do that), but learning how to write HTML, the Web’s markup language. If you don’t know how to program, writing HTML will teach you some mental habits that will help you learn. So build a home page. Try to stick to XHTML, which is a cleaner language than classic HTML. (There are good beginner tutorials on the Web; here’s one.)

But just having a home page isn’t anywhere near good enough to make you a hacker. The Web is full of home pages. Most of them are pointless, zero-content sludge — very snazzy-looking sludge, mind you, but sludge all the same (for more on this see The HTML Hell Page).

To be worthwhile, your page must have content â€” it must be interesting and/or useful to other hackers. And that brings us to the next topic…

4. If you don’t have functional English, learn it.

As an American and native English-speaker myself, I have previously been reluctant to suggest this, lest it be taken as a sort of cultural imperialism. But several native speakers of other languages have urged me to point out that English is the working language of the hacker culture and the Internet, and that you will need to know it to function in the hacker community.

Back around 1991 I learned that many hackers who have English as a second language use it in technical discussions even when they share a birth tongue; it was reported to me at the time that English has a richer technical vocabulary than any other language and is therefore simply a better tool for the job. For similar reasons, translations of technical books written in English are often unsatisfactory (when they get done at all).

Linus Torvalds, a Finn, comments his code in English (it apparently never occurred to him to do otherwise). His fluency in English has been an important factor in his ability to recruit a worldwide community of developers for Linux. It’s an example worth following.

Being a native English-speaker does not guarantee that you have language skills good enough to function as a hacker. If your writing is semi-literate, ungrammatical, and riddled with misspellings, many hackers (including myself) will tend to ignore you. While sloppy writing does not invariably mean sloppy thinking, we’ve generally found the correlation to be strong — and we have no use for sloppy thinkers. If you can’t yet write competently, learn to.

Status in the Hacker Culture

Like most cultures without a money economy, hackerdom runs on reputation. You’re trying to solve interesting problems, but how interesting they are, and whether your solutions are really good, is something that only your technical peers or superiors are normally equipped to judge.

Accordingly, when you play the hacker game, you learn to keep score primarily by what other hackers think of your skill (this is why you aren’t really a hacker until other hackers consistently call you one). This fact is obscured by the image of hacking as solitary work; also by a hacker-cultural taboo (gradually decaying since the late 1990s but still potent) against admitting that ego or external validation are involved in one’s motivation at all.

Specifically, hackerdom is what anthropologists call a gift culture. You gain status and reputation in it not by dominating other people, nor by being beautiful, nor by having things other people want, but rather by giving things away. Specifically, by giving away your time, your creativity, and the results of your skill.

There are basically five kinds of things you can do to be respected by hackers:

1. Write open-source software

The first (the most central and most traditional) is to write programs that other hackers think are fun or useful, and give the program sources away to the whole hacker culture to use.

(We used to call these works “free software”, but this confused too many people who weren’t sure exactly what “free” was supposed to mean. Most of us now prefer the term “open-source” software).

Hackerdom’s most revered demigods are people who have written large, capable programs that met a widespread need and given them away, so that now everyone uses them.

But there’s a bit of a fine historical point here. While hackers have always looked up to the open-source developers among them as our community’s hardest core, before the mid-1990s most hackers most of the time worked on closed source. This was still true when I wrote the first version of this HOWTO in 1996; it took the mainstreaming of open-source software after 1997 to change things. Today, “the hacker community” and “open-source developers” are two descriptions for what is essentially the same culture and population — but it is worth remembering that this was not always so. (For more on this, see the section called “Historical Note: Hacking, Open Source, and Free Software”.)

2. Help test and debug open-source software

They also serve who stand and debug open-source software. In this imperfect world, we will inevitably spend most of our software development time in the debugging phase. That’s why any open-source author who’s thinking will tell you that good beta-testers (who know how to describe symptoms clearly, localize problems well, can tolerate bugs in a quickie release, and are willing to apply a few simple diagnostic routines) are worth their weight in rubies. Even one of these can make the difference between a debugging phase that’s a protracted, exhausting nightmare and one that’s merely a salutary nuisance.

If you’re a newbie, try to find a program under development that you’re interested in and be a good beta-tester. There’s a natural progression from helping test programs to helping debug them to helping modify them. You’ll learn a lot this way, and generate good karma with people who will help you later on.

3. Publish useful information

Another good thing is to collect and filter useful and interesting information into web pages or documents like Frequently Asked Questions (FAQ) lists, and make those generally available.

Maintainers of major technical FAQs get almost as much respect as open-source authors.

4. Help keep the infrastructure working

The hacker culture (and the engineering development of the Internet, for that matter) is run by volunteers. There’s a lot of necessary but unglamorous work that needs done to keep it going — administering mailing lists, moderating newsgroups, maintaining large software archive sites, developing RFCs and other technical standards.

People who do this sort of thing well get a lot of respect, because everybody knows these jobs are huge time sinks and not as much fun as playing with code. Doing them shows dedication.

5. Serve the hacker culture itself

Finally, you can serve and propagate the culture itself (by, for example, writing an accurate primer on how to become a hacker :-)). This is not something you’ll be positioned to do until you’ve been around for while and become well-known for one of the first four things.

The hacker culture doesn’t have leaders, exactly, but it does have culture heroes and tribal elders and historians and spokespeople. When you’ve been in the trenches long enough, you may grow into one of these. Beware: hackers distrust blatant ego in their tribal elders, so visibly reaching for this kind of fame is dangerous. Rather than striving for it, you have to sort of position yourself so it drops in your lap, and then be modest and gracious about your status.

The Hacker/Nerd Connection

Contrary to popular myth, you don’t have to be a nerd to be a hacker. It does help, however, and many hackers are in fact nerds. Being something of a social outcast helps you stay concentrated on the really important things, like thinking and hacking.

For this reason, many hackers have adopted the label ‘geek’ as a badge of pride — it’s a way of declaring their independence from normal social expectations (as well as a fondness for other things like science fiction and strategy games that often go with being a hacker). The term ‘nerd’ used to be used this way back in the 1990s, back when ‘nerd’ was a mild pejorative and ‘geek’ a rather harsher one; sometime after 2000 they switched places, at least in U.S. popular culture, and there is now even a significant geek-pride culture among people who aren’t techies.

If you can manage to concentrate enough on hacking to be good at it and still have a life, that’s fine. This is a lot easier today than it was when I was a newbie in the 1970s; mainstream culture is much friendlier to techno-nerds now. There are even growing numbers of people who realize that hackers are often high-quality lover and spouse material.

If you’re attracted to hacking because you don’t have a life, that’s OK too — at least you won’t have trouble concentrating. Maybe you’ll get a life later on.

Points For Style

Again, to be a hacker, you have to enter the hacker mindset. There are some things you can do when you’re not at a computer that seem to help. They’re not substitutes for hacking (nothing is) but many hackers do them, and feel that they connect in some basic way with the essence of hacking.

  • Learn to write your native language well. Though it’s a common stereotype that programmers can’t write, a surprising number of hackers (including all the most accomplished ones I know of) are very able writers.
  • Read science fiction. Go to science fiction conventions (a good way to meet hackers and proto-hackers).
  • Train in a martial-arts form. The kind of mental discipline required for martial arts seems to be similar in important ways to what hackers do. The most popular forms among hackers are definitely Asian empty-hand arts such as Tae Kwon Do, various forms of Karate, Kung Fu, Aikido, or Ju Jitsu. Western fencing and Asian sword arts also have visible followings. In places where it’s legal, pistol shooting has been rising in popularity since the late 1990s. The most hackerly martial arts are those which emphasize mental discipline, relaxed awareness, and control, rather than raw strength, athleticism, or physical toughness.
  • Study an actual meditation discipline. The perennial favorite among hackers is Zen (importantly, it is possible to benefit from Zen without acquiring a religion or discarding one you already have). Other styles may work as well, but be careful to choose one that doesn’t require you to believe crazy things.
  • Develop an analytical ear for music. Learn to appreciate peculiar kinds of music. Learn to play some musical instrument well, or how to sing.
  • Develop your appreciation of puns and wordplay.

The more of these things you already do, the more likely it is that you are natural hacker material. Why these things in particular is not completely clear, but they’re connected with a mix of left- and right-brain skills that seems to be important; hackers need to be able to both reason logically and step outside the apparent logic of a problem at a moment’s notice.

Work as intensely as you play and play as intensely as you work. For true hackers, the boundaries between “play”, “work”, “science” and “art” all tend to disappear, or to merge into a high-level creative playfulness. Also, don’t be content with a narrow range of skills. Though most hackers self-describe as programmers, they are very likely to be more than competent in several related skills — system administration, web design, and PC hardware troubleshooting are common ones. A hacker who’s a system administrator, on the other hand, is likely to be quite skilled at script programming and web design. Hackers don’t do things by halves; if they invest in a skill at all, they tend to get very good at it.

Finally, a few things not to do.

  • Don’t use a silly, grandiose user ID or screen name.
  • Don’t get in flame wars on Usenet (or anywhere else).
  • Don’t call yourself a ‘cyberpunk’, and don’t waste your time on anybody who does.
  • Don’t post or email writing that’s full of spelling errors and bad grammar.

The only reputation you’ll make doing any of these things is as a twit. Hackers have long memories — it could take you years to live your early blunders down enough to be accepted.

The problem with screen names or handles deserves some amplification. Concealing your identity behind a handle is a juvenile and silly behavior characteristic of crackers, warez d00dz, and other lower life forms. Hackers don’t do this; they’re proud of what they do and want it associated with their real names. So if you have a handle, drop it. In the hacker culture it will only mark you as a loser.

Historical Note: Hacking, Open Source, and Free Software

When I originally wrote this how-to in late 1996, some of the conditions around it were very different from the way they look today. A few words about these changes may help clarify matters for people who are confused about the relationship of open source, free software, and Linux to the hacker community. If you are not curious about this, you can skip straight to the FAQ and bibliography from here.

The hacker ethos and community as I have described it here long predates the emergence of Linux after 1990; I first became involved with it around 1976, and, its roots are readily traceable back to the early 1960s. But before Linux, most hacking was done on either proprietary operating systems or a handful of quasi-experimental homegrown systems like MIT’s ITS that were never deployed outside of their original academic niches. While there had been some earlier (pre-Linux) attempts to change this situation, their impact was at best very marginal and confined to communities of dedicated true believers which were tiny minorities even within the hacker community, let alone with respect to the larger world of software in general.

What is now called “open source” goes back as far as the hacker community does, but until 1985 it was an unnamed folk practice rather than a conscious movement with theories and manifestos attached to it. This prehistory ended when, in 1985, arch-hacker Richard Stallman (“RMS”) tried to give it a name — “free software”. But his act of naming was also an act of claiming; he attached ideological baggage to the “free software” label which much of the existing hacker community never accepted. As a result, the “free software” label was loudly rejected by a substantial minority of the hacker community (especially among those associated with BSD Unix), and used with serious but silent reservations by a majority of the remainder (including myself).

Despite these reservations, RMS’s claim to define and lead the hacker community under the “free software” banner broadly held until the mid-1990s. It was seriously challenged only by the rise of Linux. Linux gave open-source development a natural home. Many projects issued under terms we would now call open-source migrated from proprietary Unixes to Linux. The community around Linux grew explosively, becoming far larger and more heterogenous than the pre-Linux hacker culture. RMS determinedly attempted to co-opt all this activity into his “free software” movement, but was thwarted by both the exploding diversity of the Linux community and the public skepticism of its founder, Linus Torvalds. Torvalds continued to use the term “free software” for lack of any alternative, but publicly rejected RMS’s ideological baggage. Many younger hackers followed suit.

In 1996, when I first published this Hacker HOWTO, the hacker community was rapidly reorganizing around Linux and a handful of other open-source operating systems (notably those descended from BSD Unix). Community memory of the fact that most of us had spent decades developing closed-source software on closed-source operating systems had not yet begun to fade, but that fact was already beginning to seem like part of a dead past; hackers were, increasingly, defining themselves as hackers by their attachments to open-source projects such as Linux or Apache.

The term “open source”, however, had not yet emerged; it would not do so until early 1998. When it did, most of hacker community adopted it within the following six months; the exceptions were a minority ideologically attached to the term “free software”. Since 1998, and especially after about 2003, the identification of ‘hacking’ with ‘open-source (and free software) development’ has become extremely close. Today there is little point in attempting to distinguish between these categories, and it seems unlikely that will change in the future.

It is worth remembering, however, that this was not always so.

Other Resources

Paul Graham has written an essay called Great Hackers, and another on Undergraduation, in which he speaks much wisdom.

There is a document called How To Be A Programmer that is an excellent complement to this one. It has valuable advice not just about coding and skillsets, but about how to function on a programming team.

I have also written A Brief History Of Hackerdom.

I have written a paper, The Cathedral and the Bazaar, which explains a lot about how the Linux and open-source cultures work. I have addressed this topic even more directly in its sequel Homesteading the Noosphere.

Rick Moen has written an excellent document on how to run a Linux user group.

Rick Moen and I have collaborated on another document on How To Ask Smart Questions. This will help you seek assistance in a way that makes it more likely that you will actually get it.

If you need instruction in the basics of how personal computers, Unix, and the Internet work, see The Unix and Internet Fundamentals HOWTO.

When you release software or write patches for software, try to follow the guidelines in the Software Release Practice HOWTO.

If you enjoyed the Zen poem, you might also like Rootless Root: The Unix Koans of Master Foo.

Frequently Asked Questions

Q: How do I tell if I am already a hacker?
Q: Will you teach me how to hack?
Q: How can I get started, then?
Q: When do you have to start? Is it too late for me to learn?
Q: How long will it take me to learn to hack?
Q: Is Visual Basic a good language to start with?
Q: Would you help me to crack a system, or teach me how to crack?
Q: How can I get the password for someone else’s account?
Q: How can I break into/read/monitor someone else’s email?
Q: How can I steal channel op privileges on IRC?
Q: I’ve been cracked. Will you help me fend off further attacks?
Q: I’m having problems with my Windows software. Will you help me?
Q: Where can I find some real hackers to talk with?
Q: Can you recommend useful books about hacking-related subjects?
Q: Do I need to be good at math to become a hacker?
Q: What language should I learn first?
Q: What kind of hardware do I need?
Q: I want to contribute. Can you help me pick a problem to work on?
Q: Do I need to hate and bash Microsoft?
Q: But won’t open-source software leave programmers unable to make a living?
Q: Where can I get a free Unix?
Q: How do I tell if I am already a hacker?
A: Ask yourself the following three questions:

  • Do you speak code, fluently?
  • Do you identify with the goals and values of the hacker community?
  • Has a well-established member of the hacker community ever called you a hacker?

If you can answer yes to all three of these questions, you are already a hacker. No two alone are sufficient.

The first test is about skills. You probably pass it if you have the minimum technical skills described earlier in this document. You blow right through it if you have had a substantial amount of code accepted by an open-source development project.

The second test is about attitude. If the five principles of the hacker mindset seemed obvious to you, more like a description of the way you already live than anything novel, you are already halfway to passing it. That’s the inward half; the other, outward half is the degree to which you identify with the hacker community’s long-term projects.

Here is an incomplete but indicative list of some of those projects: Does it matter to you that Linux improve and spread? Are you passionate about software freedom? Hostile to monopolies? Do you act on the belief that computers can be instruments of empowerment that make the world a richer and more humane place?

But a note of caution is in order here. The hacker community has some specific, primarily defensive political interests — two of them are defending free-speech rights and fending off “intellectual-property” power grabs that would make open source illegal. Some of those long-term projects are civil-liberties organizations like the Electronic Frontier Foundation, and the outward attitude properly includes support of them. But beyond that, most hackers view attempts to systematize the hacker attitude into an explicit political program with suspicion; we’ve learned, the hard way, that these attempts are divisive and distracting. If someone tries to recruit you to march on your capitol in the name of the hacker attitude, they’ve missed the point. The right response is probably â€œShut up and show them the code.”

The third test has a tricky element of recursiveness about it. I observed in the section called “What Is a Hacker?” that being a hacker is partly a matter of belonging to a particular subculture or social network with a shared history, an inside and an outside. In the far past, hackers were a much less cohesive and self-aware group than they are today. But the importance of the social-network aspect has increased over the last thirty years as the Internet has made connections with the core of the hacker subculture easier to develop and maintain. One easy behavioral index of the change is that, in this century, we have our own T-shirts.

Sociologists, who study networks like those of the hacker culture under the general rubric of “invisible colleges”, have noted that one characteristic of such networks is that they have gatekeepers — core members with the social authority to endorse new members into the network. Because the “invisible college” that is hacker culture is a loose and informal one, the role of gatekeeper is informal too. But one thing that all hackers understand in their bones is that not every hacker is a gatekeeper. Gatekeepers have to have a certain degree of seniority and accomplishment before they can bestow the title. How much is hard to quantify, but every hacker knows it when they see it.

Q: Will you teach me how to hack?
A: Since first publishing this page, I’ve gotten several requests a week (often several a day) from people to “teach me all about hacking”. Unfortunately, I don’t have the time or energy to do this; my own hacking projects, and working as an open-source advocate, take up 110% of my time.

Even if I did, hacking is an attitude and skill you basically have to teach yourself. You’ll find that while real hackers want to help you, they won’t respect you if you beg to be spoon-fed everything they know.

Learn a few things first. Show that you’re trying, that you’re capable of learning on your own. Then go to the hackers you meet with specific questions.

If you do email a hacker asking for advice, here are two things to know up front. First, we’ve found that people who are lazy or careless in their writing are usually too lazy and careless in their thinking to make good hackers — so take care to spell correctly, and use good grammar and punctuation, otherwise you’ll probably be ignored. Secondly, don’t dare ask for a reply to an ISP account that’s different from the account you’re sending from; we find people who do that are usually thieves using stolen accounts, and we have no interest in rewarding or assisting thievery.

Q: How can I get started, then?
A: The best way for you to get started would probably be to go to a LUG (Linux user group) meeting. You can find such groups on the LDP General Linux Information Page; there is probably one near you, possibly associated with a college or university. LUG members will probably give you a Linux if you ask, and will certainly help you install one and get started.
Q: When do you have to start? Is it too late for me to learn?
A: Any age at which you are motivated to start is a good age. Most people seem to get interested between ages 15 and 20, but I know of exceptions in both directions.
Q: How long will it take me to learn to hack?
A: That depends on how talented you are and how hard you work at it. Most people who try can acquire a respectable skill set in eighteen months to two years, if they concentrate. Don’t think it ends there, though; in hacking (as in many other fields) it takes about ten years to achieve mastery. And if you are a real hacker, you will spend the rest of your life learning and perfecting your craft.
Q: Is Visual Basic a good language to start with?
A: If you’re asking this question, it almost certainly means you’re thinking about trying to hack under Microsoft Windows. This is a bad idea in itself. When I compared trying to learn to hack under Windows to trying to learn to dance while wearing a body cast, I wasn’t kidding. Don’t go there. It’s ugly, and it never stops being ugly.

There is a specific problem with Visual Basic; mainly that it’s not portable. Though there is a prototype open-source implementations of Visual Basic, the applicable ECMA standards don’t cover more than a small set of its programming interfaces. On Windows most of its library support is proprietary to a single vendor (Microsoft); if you aren’t extremely careful about which features you use — more careful than any newbie is really capable of being — you’ll end up locked into only those platforms Microsoft chooses to support. If you’re starting on a Unix, much better languages with better libraries are available. Python, for example.

Also, like other Basics, Visual Basic is a poorly-designed language that will teach you bad programming habits. No, don’t ask me to describe them in detail; that explanation would fill a book. Learn a well-designed language instead.

One of those bad habits is becoming dependent on a single vendor’s libraries, widgets, and development tools. In general, any language that isn’t fully supported under at least Linux or one of the BSDs, and/or at least three different vendors’ operating systems, is a poor one to learn to hack in.

Q: Would you help me to crack a system, or teach me how to crack?
A: No. Anyone who can still ask such a question after reading this FAQ is too stupid to be educable even if I had the time for tutoring. Any emailed requests of this kind that I get will be ignored or answered with extreme rudeness.
Q: How can I get the password for someone else’s account?
A: This is cracking. Go away, idiot.
Q: How can I break into/read/monitor someone else’s email?
A: This is cracking. Get lost, moron.
Q: How can I steal channel op privileges on IRC?
A: This is cracking. Begone, cretin.
Q: I’ve been cracked. Will you help me fend off further attacks?
A: No. Every time I’ve been asked this question so far, it’s been from some poor sap running Microsoft Windows. It is not possible to effectively secure Windows systems against crack attacks; the code and architecture simply have too many flaws, which makes securing Windows like trying to bail out a boat with a sieve. The only reliable prevention starts with switching to Linux or some other operating system that is designed to at least be capable of security.
Q: I’m having problems with my Windows software. Will you help me?
A: Yes. Go to a DOS prompt and type “format c:”. Any problems you are experiencing will cease within a few minutes.
Q: Where can I find some real hackers to talk with?
A: The best way is to find a Unix or Linux user’s group local to you and go to their meetings (you can find links to several lists of user groups on the LDP site at ibiblio).

(I used to say here that you wouldn’t find any real hackers on IRC, but I’m given to understand this is changing. Apparently some real hacker communities, attached to things like GIMP and Perl, have IRC channels now.)

Q: Can you recommend useful books about hacking-related subjects?
A: I maintain a Linux Reading List HOWTO that you may find helpful. The Loginataka may also be interesting.

For an introduction to Python, see the tutorial on the Python site.

Q: Do I need to be good at math to become a hacker?
A: No. Hacking uses very little formal mathematics or arithmetic. In particular, you won’t usually need trigonometry, calculus or analysis (there are exceptions to this in a handful of specific application areas like 3-D computer graphics). Knowing some formal logic and Boolean algebra is good. Some grounding in finite mathematics (including finite-set theory, combinatorics, and graph theory) can be helpful.

Much more importantly: you need to be able to think logically and follow chains of exact reasoning, the way mathematicians do. While the content of most mathematics won’t help you, you will need the discipline and intelligence to handle mathematics. If you lack the intelligence, there is little hope for you as a hacker; if you lack the discipline, you’d better grow it.

I think a good way to find out if you have what it takes is to pick up a copy of Raymond Smullyan’s book What Is The Name Of This Book?. Smullyan’s playful logical conundrums are very much in the hacker spirit. Being able to solve them is a good sign; enjoying solving them is an even better one.

Q: What language should I learn first?
A: XHTML (the latest dialect of HTML) if you don’t already know it. There are a lot of glossy, hype-intensive bad HTML books out there, and distressingly few good ones. The one I like best is HTML: The Definitive Guide.

But HTML is not a full programming language. When you’re ready to start programming, I would recommend starting with Python. You will hear a lot of people recommending Perl, but it’s harder to learn and (in my opinion) less well designed.

C is really important, but it’s also much more difficult than either Python or Perl. Don’t try to learn it first.

Windows users, do not settle for Visual Basic. It will teach you bad habits, and it’s not portable off Windows. Avoid.

Q: What kind of hardware do I need?
A: It used to be that personal computers were rather underpowered and memory-poor, enough so that they placed artificial limits on a hacker’s learning process. This stopped being true in the mid-1990s; any machine from an Intel 486DX50 up is more than powerful enough for development work, X, and Internet communications, and the smallest disks you can buy today are plenty big enough.

The important thing in choosing a machine on which to learn is whether its hardware is Linux-compatible (or BSD-compatible, should you choose to go that route). Again, this will be true for almost all modern machines. The only really sticky areas are modems and wireless cards; some machines have Windows-specific hardware that won’t work with Linux.

There’s a FAQ on hardware compatibility; the latest version is here.

Q: I want to contribute. Can you help me pick a problem to work on?
A: No, because I don’t know your talents or interests. You have to be self-motivated or you won’t stick, which is why having other people choose your direction almost never works.

Try this. Watch the project announcements scroll by on Freshmeat for a few days. When you see one that makes you think “Cool! I’d like to work on that!”, join it.

Q: Do I need to hate and bash Microsoft?
A: No, you don’t. Not that Microsoft isn’t loathsome, but there was a hacker culture long before Microsoft and there will still be one long after Microsoft is history. Any energy you spend hating Microsoft would be better spent on loving your craft. Write good code — that will bash Microsoft quite sufficiently without polluting your karma.
Q: But won’t open-source software leave programmers unable to make a living?
A: This seems unlikely — so far, the open-source software industry seems to be creating jobs rather than taking them away. If having a program written is a net economic gain over not having it written, a programmer will get paid whether or not the program is going to be open-source after it’s done. And, no matter how much “free” software gets written, there always seems to be more demand for new and customized applications. I’ve written more about this at the Open Source pages.
Q: Where can I get a free Unix?
A: If you don’t have a Unix installed on your machine yet, elsewhere on this page I include pointers to where to get the most commonly used free Unix. To be a hacker you need motivation and initiative and the ability to educate yourself. Start now…
Thanks Kj Derricks