Penetration Testing

Attacks can be phishing or brute force and widely dispersed. They can be precise, patient and disciplined. The attacker's motivation and preparation level of the adversary will determine the strategy.

Rules of engagement, deadlines or approved tools are not of importance to the attacker and they do not confine themselves with these. They do not have respect for intellectual property and don't dismiss an opportunity because it's been done before. What they appreciate are administrators who practice careless security habits so they, the attackers can get in and out quickly and then move on to the next challenge. Attackers will be imaginative and resourceful when the old tricks aren't working.

Attackers do not keep regular hours, nor do they refuse to work weekends or holidays and rarely complain about their conditions. They are resourceful. They can work from airports, parking lots, coffee houses, classrooms, dumpsters or cubicles.

Reasons for Penetration Tests

Penetration tests can range from a simple test of a single vulnerability to a complete engagement lasting for months. A clear business objective is important when ordering any penetration tests. Without this clear objective, the scope of the test will not be defined and the appropriate documents that charter the project cannot be drafted.

There are a number of reasons why penetration tests might be performed. Included are:

• Risk Management Data Research
• Compliance
• Incident Response Plans Testing
• False Positive Verification

Risk Management Data Research

Risk Management is an ongoing process in an organization. Defenses already in place must be considered when threats are being analyzed. Feeling that it does not have sufficient or accurate data on the true effectiveness of a risk plan, they may request a test to determine the true exposure level of a possible weakness.

For example, a Web application might be protected by a Web Application Firewall (WAF) that the salesman said would block all Cross-Site Scripting (XSS) attacks. Wisely, a suggestion that the risk cannot be considered 100 percent mitigated until a complete test of the web application could be conducted.

As a result of the test, it is shown that the WAF is effective against some threats but partially effective against one. A plan is created to fix the application itself instead of relying on the WAF.

Compliance

Assessment and penetration tests are part of the compliance process. The are invaluable tools for the measure of both the current state of security and the effectiveness of configuration process changes that are made in order to achieve the required security goals. Simply, assessments and penetration tests are a way to test policies and policy enforcement.

Incident Response Plans Testing

Plans are worthless unless they are tested. The core responsibilities of any organization are incident responses, business continuity planning and disaster recovery planning. For companies that are publicly traded, negligence issues can result if there is a lack of effort in this area.

False Positive Verification

Considered at times only rough sketches of the current state of security levels, assessments are a necessary test within a network. Ofter here are questions to whether or not these assessments show critical vulnerabilities or simply report false positives, that is, something resembles a vulnerability but is not.

There are three issues with vulnerability scanner assessment tools. They are:

• They produce a lot of noise on the network.
• They do not perform vulnerability linkage.
• They are considered prone to false positives.

Vulnerability scanners send specially crafted traffic to target hosts and then analyzes the responses. Intrusion detection system alarms are triggered and the possibility to cause Denial of Service (DoS) on a network can result as these tests can prove to be dangerous.

To demonstrate a confirmed exposure, some vulnerabilities on the network require multiple steps. For example, gaining access to one machine only to use it to gain access to another. Tools such as Metaspoit, Canvas and Core Impact can be used to take these scans to this deep level as general vulnerability scans can not.

Having improved over the years, scanning tools do not live up to the old reputation of being prone to false positives. More often, the assessment is correct and the remediation of the vulnerability is required or advised.

Types of Penetration Tests

Several different ways are used to categorize the tests and descriptions of penetration testing. Importantly, you must select the correct type of test for the needs of the client. The three types of testing are:

• Announced vs. Unannounced Tests
• Internal vs. External Tests
• Black Box vs. Grey Box vs. White Box Tests

Announced vs. Unannounced Tests

Tests that are announced occur when everyone, especially the administrators of the network are aware that the test is being conducted. Being caught is of no concern nor is it part of the analysis of the results. Typically, maintenance windows are set aside for these tests to minimize the effects of the test on resources and services.

Unannounced tests occur when the monitoring and incident response plans are being assessed. Only the project sponsor and system owners will know about the test being conducted. The responsibility of this test resides on the sponsor of the project should the tester get caught.

Internal vs. External Tests

The position of the simulated attacker can be either internal or external. At times, the major goal of the test is to start from the outside in an attempt to find weaknesses by infiltrating the organization. From there, the simulated attack attempts to find internal weaknesses.

White Box vs. Black Box vs. Grey Box Tests

A White Box test is also known as Blue Teaming and is typically more of an assessment type of test. Information gathering steps are skipped in an effort to save both time and money.

A Black Box test is also known as Red Teaming and are more time consuming and costly. These tests simulate something much closer to the true attack scenario.

A Grey Box test have a limited scope. A clear objective is given to the tester, however all of the information required to achieve the objective may be withheld. As an example, testing, or "auditing" user accounts for passwords that an attacker might obtain. This can be done via social engineering, obtaining local hashes, sniffing hashes, guessing, using fake websites and so on.

The Basic Steps of Engagement

The best way to view the penetration test process is by modifying the attacks to fit a format penetration test. Here are the basic steps for this engagement.

• Initiate the Engagement
• Project Scope and Charter
• Reconnaissance
• Scanning
  • Unearth Initial Information
  • Locate the Network Range
  • Ascertain Active Machines
  • Discover Open Ports and Access Points
  • Detect Operating Systems
  • Uncover Services on Ports
  • Map the Network
• Vulnerability Assessment
• Gaining Access
• Ending the Test

When performing a defense-oriented test, it is unacceptable to maintain access or to gain complete control over a system. This decision is a matter of agreement between the client and tester where expectations must be clear and in writing.

The type of testing change depending on the type of test ordered. Most importantly is to not breach the rules of engagement set, to test systems that have not been given permission to test or to violate the confidentiality of data. Denial of service should be avoided though this is an inherent risk that the client must be made aware of and will agree to in writing. The objective of the test is to prove that a vulnerability exists and to document supporting facts that explain how it was discovered, why it matters and how it can be resolved.

Setting Up A Staging Lab

Having the ability to stage an environment for practice and analysis of tools is a skill critical for any penetration tester. Two methods are available. Some prefer only real hardware and gather a collection of computers and routers for this method while others prefer a virtualization. For this method, virtualization such as VMWare Workstation or the free, open source Virtualbox.

The testing lab should contain various operating systems within various stages of patch level and configuration. With virtualization, a tester can replace parts of a target network to test out scenarios in advance or analyze their tools with a packet sniffer before using them just to assure they aren't making more noise than intended.

Setting Up the Engagement

Planning for the correct test important when setting up the penetration test for the client. The engineer must advise the customer on what type of test will fit the situation and provide a clear explanation of the expectations and exit criteria. Once established, the penetration team needs to be assembled.

Small engagements might be handled by a single person using a wide range of advanced skills. By using sophisticated penetration testing tools that go beyond the assessment level, both time and cost can be kept to a minimum. This type of test is usually an insider announced test as the tools can generate alerts in the intrusion detection agents. With the coordination of the incident response team, these scans should be run during a maintenance window.

The use of a team of professionals are usually gathered for larger tests. A mixture of skills that cover the whole OSI model should be present. There should be a physical tester, a social engineer, a network and scanning expert, a rather creative programmer, researchers and importantly, a customer liaison. The liaison is usually the only team member that the client ever actually meets. The liaison must be comfortable in both business and technical discussions.

Planning and Management

Regardless of size, the professional penetration test is managed as a project. From both a metrics and control perspective, it must be successful and repeatable. A business abstraction of the real experiences of the testers must be improvised and creative. Though counter intuitive as the relationship might seem at times, it does work because of both the technical talent and business processes necessary.

The topic of project management is far to vast and wide to be discussed here. These are the basic distinct phases that must be considered when planning a penetration test.

• Initiation
• Planning
• Execution
• Control
• Closure

Initiation

There must be a reason for the project. An understanding of what effort and impact tolerance that this will take to accomplish the project must be in place as well. This is where the initiation stage comes into play.

The initiation stage is about the commitment to the test. There is a definition of the problem and resolution. Also discussed are the time estimates, financial budgets and impact of the surrounding operations. This is where all emotional and political support are gathered. A sponsor will step up to fund the idea and take responsibility.

The charter will be signed to launch the project. This charter will include a clear definition of exactly what the sponsor, (also sometimes known as the customer) wants. There are times that both the sponsor and customer will be the same person.

Planning

Failure at this point is not an option as the project has begun. There will be no excuses built into the plan. Only contingencies and risk management options will be accepted. The project definition will be broken down into manageable and assignable parts. The schedules will be calculated and the assignments will be delegated.

Execution

This is where the plan is in progress. Best practice here is to follow the plan and to make good notes along it's course. It is almost impossible to accurately document the plan after too many things have happened. Remember, not all penetration tests result in a breech. The report is what the customer will use to determine if the efforts they paid for were provided or not.

Control

Being able to make adjustments that are well informed and not counterproductive to other members of the team is what control is all about. It is about handling the unexpected, the unknowns in such a manner that they are barely noticeable. This phase develops and maintains the confidence of both the sponsor and client. It lets them know what is going on and keeps the project on track.

There will be the inevitable status report requests. An answer of "I don't know" only causes the client to lose confidence in the team. The team members must be able to explain at all times what they are doing and why they are doing it. Detailed documentation should be kept showing what has already been done and will eventually be developed into the final project report.

Closure

This is where the deliverables, or also known as the exit criteria of the project have been met. A report entitled the Administrative Close is a part of this phase and will include all formal acceptance of the results and agreements to support regarding of clearing all tracks of the tests and the payment.

A meeting of the penetration testing team should be held to discuss lessons learned and review the project as a whole.

Initial Documentation

Agreements made in writing and the understanding between the client and the tester should be made and be perfectly clear before the execution and monitoring phases of the engagement begin. The final outcome of the test depend on this. The has the potential of being too dangerous to allow any sloppiness in regards to the cooperation that must exist between the sponsor and the penetration team.

These are the core doucments that must make up the initiation phase of the penetration test.

• Project Scope
• Indemnity Clause
• "Get out of jail free" Card
• Rules of Engagement
• Communications Plan
• Non-Disclosure Agreements
• Exit Criteria

Project Scope

The Project Scope is a document of high-level view defining the test. It provides the start and end dates, budget estimates and a list of the people that will be participating as part of the team. If any member of the team change during the engagement, the client should be informed immediately.

The Statement of Work (SOW) and Project Charter are associated with this as well. This document is the point that marks the official beginning of the engagement and from this point going forward, failing to finish the task is not an option.

Indemnity Clause

The Indemnity Clause is a legal statement protecting the tester should something go wrong. The client is completely and fully informed of all risks prior to the start of the penetration test. This document makes the client responsible for the decision to move forward and releases the penetration team from liability.

"Get out of Jail Free" Card

The person representing the client and is the project sponsor of the penetration test is authorized to pay for the project. Should the penetration test be unannounced, there is a good chance that the tester could get caught. If this happens, it is the responsibility of the sponsor to step in and reveal that this exercise was approved and that all responsibility lies with the decision maker.

Rules of Engagement

This document restrains the testing team, protects the client and makes sure that none of the activities of the test will exceed the comfort zone of the project sponsor. There is a comprehensive checklist of activities that is negotiated. This is thorough and complete as to what tools and techniques the team is allowed to use and explicitly not allowed to use.

Some examples are, social engineering may be allowed over the phone but not directly in person. DoS tools are not allowed at all, Rootkits, viral code and illicit servers obtained from unknown sources may not be permitted as they can not be completely controlled or cleaned up by the testing team.

Should the team write scripts during the test, is the script considered proprietary intellectual property of the testers or does all of the source code need to be disclosed to the client? The rules of engagement is vital to the outcome of the test and the team must remain within these boundaries if other legal protections are to be held.

Communications Plan

At times a testing team may encounter a vulnerability that requires immediate attention as the risk is so great to the client that the reporting of this vulnerability can not wait until the end of the project. For this, there must be a proper procedure in place. At the same time, should the testers affect a service and have to stop what they are doing, there must be a way the customer can contact the team and ask for a pause. This usually involves providing an explanation as well.

The communication plan has to fit within the project scope and type of test that has been ordered by the client. If there is too much communication, it might not only be dangerous but might also diminish the confidence in the testers themselves. Needed is a skilled liaison working with the team and client. This liaison must exercise good judgment in this area and must wear all of these hats.

Non-Disclosure Agreements

The results of the test must be tightly controlled. There has to be a method of proving the report that has to be agreed upon as well as to the proper destruction of any copies of the notes taken along the way.

The client understandably wants the assurance that the information gathered by the penetration testers is considered sacred and protected. The company providing the testing services wants referrals and positive feedback, yet they can not always reveal their clients due to the nature of this sensitive information.

The value of the Licensed Penetration Tester (LPT) is a privately held license from the EC-Council. When experience can not be explicitly discussed, it is helpful when a baseline of knowledge can be communicated and an organization can back up the endeavor undertaken to achieve the credential.

Exit Criteria

The Exit Criteria contains the report and a follow-up discussion regarding a schedule of future tests. The project must have a defined end but must establish with the client what the next step should be. The next project might be a routine repeat of the previous one or it may become a whole new service based on the results of the current test now being closed.

This closing meeting of the exit criteria is both an end to the current project and a milestone in a larger scope of things to come. It is a chance to draft a new charter and start all over again.

Style and Tone

The major part of the exit criteria is the report. It helps the client reach the objective that caused them to order the test to begin with. It is also a lasting impression clients will have on the work of the testing team and is written in business language; the wording being straightforward and to the point.

If problems are revealed, make sure the explanation of the issues are dry, empirical and fact-based. Do not insert personal opinions and of course never be offensive. Refrain from making the report read like a resume reflecting the work of the team and do not use words like "I" or "we". Make the report a source of information where the objective of the test being the object, not the testers themselves.

Many ways can be used to arrange the information delivered. The customer might even have a preference and this preference should be communicated in the project plan. The summary of information should be accurate but at the same time easily digestible. Make issues found during the test sound manageable. Fear, uncertainty and doubt have no place in the final report; it should be the exact opposite. The report should make the client feel that any problems discovered are manageable and state clearly what needs to be addressed.

Should the test be uneventful, realize that gaping holes in the network were not necessarily expected. Sometimes clients are on top of things with their network security and the test will show this. Free testing services should never be given away nor should feelings of guilt be allowed for not finding vulnerabilities. The rules of engagement should never be violated in the frustration over not finding anything to put in the report.

The best opportunity to demonstrate the value of the test is in the proper documentation and justification of each action during the test. The report should honestly and accurately explain to the client what took place. The ultimate decision about whether or not the proper effort and diligence was performed lays with the client.

Elements to Include in the Report

Report writing is a task that is a true skill that is harder than most people think and is a medium in which how the entire effort of the team is judged. It can also make the difference between repeat business or not; whether a client believes they received the product and service they paid for.

The report should contain the following six elements for it to be effective.

• Executive summary
• Definitions and conventions
• Vulnerability rated on a critical or accepted scoring system
• Index numbers of industry recognized vulnerability databases, if applicable
• Description of the vulnerabilities found or activities conducted while searching for them
• Summary and conclusion

Executive Summary

The Executive Summary is what tells the readers what they are reading about and what the conclusion is. It is where you acknowledge the audience and thank them for their time. Make this report as compelling as possible as it will then be read with interest and critical thought.

If the executive summary is not compelling and dull, it will most likely be discarded no matter how good the information is and nobody will read it.

Definitions and Conventions

Technical jargon should be avoided if at all possible, but at times this is not possible. Most of the time it is better to just use the term and move on. However, a brief description as to what the term refers to will help the reader better understand the meaning of the jargon used.

Vulnerability Rated on a Critical or Accepted Scoring System

A metric should be assigned to all discovered vulnerabilities. This helps the consultant and client collaborate on a set of priorities. Each issue should be assigned a score that can help in this prioritization. This is subjective and all assessment tools provide a system of their own.

The scoring scale should be explained in the Definitions and Conventions section of the report.

Should you decided on a scale of 1 to 10, tell the client exactly what "1" represents and what "10" represents. You should also describe exactly what all of the numbers in between represent as well. Keep all definitions straightforward.

Some vulnerability assessment tools have a built in scoring system. Also available are tools called Common Vulnerability Scoring Systems (CVSS) that take into account several factors relating to true risk. A description and more information about CVSS can be found on the Common Vulnerability Scoring System SIG site.

Index Numbers of Industry Recognized Vulnerability Databases, if applicable

The report that the customer receives should be very informative as far about the vulnerabilities, what their causes were, how to reproduce them if necessary and how to fix them. Should the vulnerability be indexed in one of the industry databases, provide the index number and link in the report so that the customer can do further research on their own.

If the test was a black box style, focus on what steps were taken to lead to the discovery of the vulnerability. Allow the customer to choose to research further using your references if they desire to. Make sure the report is written in "business language."

If the test was a white box style, it is acceptable to use the reports your assessment tools created automatically. Add value to your report that will service the client specifically. Customize your report to your customer's specific needs and system. Do not give the customer a stock report leaving them felt that they have spent good money on something they could have done themselves.

Description of the Vulnerabilities Found or Activities Conducted While Searching for Them

The title of the vulnerability report should read like a newspaper headline, short but an attention getter.

Keep the reader focused on the risk and not on the tester. Emphasize the reason why the vulnerability is important. Make sure it is described as to how the weakness was found including the tools and actions used. It is important to keep the flow of the text going from one topic to the other. If there are details and screen-shots, keep these confined to an appendix. Give your customer the choice to pursue further details or to just take your word for your findings. Continue this process throughout the whole report.

Be sure that there is enough evidence to be convincing and provide guidance to an experienced administrator in terms of reproducing the problem. This should be done without creating a complete tutorial of the issue, and it must be done in business language while having substance at the same time.

Summary and Conclusion

Summary and conclusion reports should not contain personal opinion. It is vital to accurately state the assessments found to the client. It is unacceptable to insult, berate or bully the client with statements such as, for example, "This is a poorly designed network and shows a lack of competency. This is why there were so many vulnerabilities were discovered." Acceptable reporting should reflect this thought with a more professional, non-opinionated statement, such as, "The testing team has discovered several vulnerabilities that require immediate attention."

Delivering the Report

These reports are critical and should not be seen or end up in the hands of the wrong people, therefore special arrangements should be made as to how the report is delivered.

At times a document that is password protected is acceptable, like a password protected PDF (.pdf) document, however, the passwords on these type of documents are not that difficult to crack. Should a PDF document be used, it is important to state how long the testing team will bear the responsibility of the security of these documents. The customer should be advised that the team will keep the document for a set period of time and then the document will be destroyed. This report may be all that is left of a penetration test that cost the customer thousands of dollars.

Another option is to make a single printout of the report in the customer's building and hand-deliver the report directly to the project sponsor. A debriefing may be asked so that an explanation of the report can be given. If a PowerPoint is created for this meeting, both the document and PowerPoint should be properly handled and destroyed.