Introduction
Attacks can be phishing or brute force and widely dispersed. They can be precise, patient and disciplined. The attacker's
motivation and preparation level of the adversary will determine the strategy.
Rules of engagement, deadlines or approved tools are not of importance to the attacker and they do not confine themselves
with these. They do not have respect for intellectual property and don't dismiss an opportunity because it's been done
before. What they appreciate are administrators who practice careless security habits so they, the attackers can get in
and out quickly and then move on to the next challenge. Attackers will be imaginative and resourceful when the old tricks
aren't working.
Attackers do not keep regular hours, nor do they refuse to work weekends or holidays and rarely complain about their
conditions. They are resourceful. They can work from airports, parking lots, coffee houses, classrooms, dumpsters or
cubicles.
General Penetration Testing Methodologies
Reasons for Penetration Tests
Penetration tests can range from a simple test of a single vulnerability to a complete engagement lasting for months. A
clear business objective is important when ordering any penetration tests. Without this clear objective, the scope of the
test will not be defined and the appropriate documents that charter the project cannot be drafted.
There are a number of reasons why penetration tests might be performed. Included are:
- Risk Management Data Research
- Compliance
- Incident Response Plans Testing
- False Positive Verification
Risk Management Data Research
Risk Management is an ongoing process in an organization. Defenses already in place must be considered when threats
are being analyzed. Feeling that it does not have sufficient or accurate data on the true effectiveness of a risk
plan, they may request a test to determine the true exposure level of a possible weakness.
For example, a Web application might be protected by a Web Application Firewall (WAF) that the salesman said
would block all Cross-Site Scripting (XSS) attacks. Wisely, a suggestion that the risk cannot be considered 100
percent mitigated until a complete test of the web application could be conducted.
As a result of the test, it is shown that the WAF is effective against some threats but partially effective against one.
A plan is created to fix the application itself instead of relying on the WAF.
Compliance
Assessment and penetration tests are part of the compliance process. The are invaluable tools for the measure of both
the current state of security and the effectiveness of configuration process changes that are made in order to achieve
the required security goals. Simply, assessments and penetration tests are a way to test policies and policy enforcement.
Incident Response Plans Testing
Plans are worthless unless they are tested. The core responsibilities of any organization are incident responses,
business continuity planning and disaster recovery planning. For companies that are publicly traded, negligence
issues can result if there is a lack of effort in this area.
False Positive Verification
Considered at times only rough sketches of the current state of security levels, assessments are a necessary test within
a network. Ofter here are questions to whether or not these assessments show critical vulnerabilities or simply report
false positives, that is, something resembles a vulnerability but is not.
There are three issues with vulnerability scanner assessment tools. They are:
- They produce a lot of noise on the network.
- They do not perform vulnerability linkage.
- They are considered prone to false positives.
Vulnerability scanners send specially crafted traffic to target hosts and then analyzes the responses. Intrusion
detection system alarms are triggered and the possibility to cause Denial of Service (DoS) on a network can result as
these tests can prove to be dangerous.
To demonstrate a confirmed exposure, some vulnerabilities on the network require multiple steps. For example, gaining
access to one machine only to use it to gain access to another. Tools such as Metaspoit, Canvas and Core Impact can
be used to take these scans to this deep level as general vulnerability scans can not.
Having improved over the years, scanning tools do not live up to the old reputation of being prone to false positives.
More often, the assessment is correct and the remediation of the vulnerability is required or advised.
Types of Penetration Tests
Several different ways are used to categorize the tests and descriptions of penetration testing. Importantly, you must
select the correct type of test for the needs of the client. The three types of testing are:
- Announced vs. Unannounced Tests
- Internal vs. External Tests
- Black Box vs. Grey Box vs. White Box Tests
Announced vs. Unannounced Tests
Tests that are announced occur when everyone, especially the administrators of the network are aware that the test is
being conducted. Being caught is of no concern nor is it part of the analysis of the results. Typically, maintenance
windows are set aside for these tests to minimize the effects of the test on resources and services.
Unannounced tests occur when the monitoring and incident response plans are being assessed. Only the project sponsor
and system owners will know about the test being conducted. The responsibility of this test resides on the sponsor of
the project should the tester get caught.
Internal vs. External Tests
The position of the simulated attacker can be either internal or external. At times, the major goal of the test is to
start from the outside in an attempt to find weaknesses by infiltrating the organization. From there, the simulated
attack attempts to find internal weaknesses.
White Box vs. Black Box vs. Grey Box Tests
A White Box test is also known as Blue Teaming and is typically more of an assessment type of test. Information
gathering steps are skipped in an effort to save both time and money.
A Black Box test is also known as Red Teaming and are more time consuming and costly. These tests simulate
something much closer to the true attack scenario.
A Grey Box test have a limited scope. A clear objective is given to the tester, however all of the information required
to achieve the objective may be withheld. As an example, testing, or "auditing" user accounts for passwords that an
attacker might obtain. This can be done via social engineering, obtaining local hashes, sniffing hashes, guessing,
using fake websites and so on.
The Basic Steps of Engagement
The best way to view the penetration test process is by modifying the attacks to fit a format penetration test. Here are
the basic steps for this engagement.
- Initiate the Engagement
- Project Scope and Charter
- Reconnaissance
- Scanning
- Unearth Initial Information
- Locate the Network Range
- Ascertain Active Machines
- Discover Open Ports and Access Points
- Detect Operating Systems
- Uncover Services on Ports
- Map the Network
- Vulnerability Assessment
- Gaining Access
- Ending the Test
When performing a defense-oriented test, it is unacceptable to maintain access or to gain complete control over a system.
This decision is a matter of agreement between the client and tester where expectations must be clear and in writing.
The type of testing change depending on the type of test ordered. Most importantly is to not breach the rules of
engagement set, to test systems that have not been given permission to test or to violate the confidentiality of data.
Denial of service should be avoided though this is an inherent risk that the client must be made aware of and will
agree to in writing. The objective of the test is to prove that a vulnerability exists and to document supporting
facts that explain how it was discovered, why it matters and how it can be resolved.
Setting Up A Staging Lab
Having the ability to stage an environment for practice and analysis of tools is a skill critical for any penetration
tester. Two methods are available. Some prefer only real hardware and gather a collection of computers and routers for
this method while others prefer a virtualization. For this method, virtualization such as VMWare Workstation or
the free, open source Virtualbox.
The testing lab should contain various operating systems within various stages of patch level and configuration. With
virtualization, a tester can replace parts of a target network to test out scenarios in advance or analyze their tools
with a packet sniffer before using them just to assure they aren't making more noise than intended.
Requirements of an Engagement
Setting Up the Engagement
Planning for the correct test important when setting up the penetration test for the client. The engineer must advise
the customer on what type of test will fit the situation and provide a clear explanation of the expectations and exit
criteria. Once established, the penetration team needs to be assembled.
Small engagements might be handled by a single person using a wide range of advanced skills. By using sophisticated
penetration testing tools that go beyond the assessment level, both time and cost can be kept to a minimum. This type of
test is usually an insider announced test as the tools can generate alerts in the intrusion detection agents. With the
coordination of the incident response team, these scans should be run during a maintenance window.
The use of a team of professionals are usually gathered for larger tests. A mixture of skills that cover the whole OSI
model should be present. There should be a physical tester, a social engineer, a network and scanning expert, a rather
creative programmer, researchers and importantly, a customer liaison. The liaison is usually the only team member that
the client ever actually meets. The liaison must be comfortable in both business and technical discussions.
Planning and Management
Regardless of size, the professional penetration test is managed as a project. From both a metrics and control perspective,
it must be successful and repeatable. A business abstraction of the real experiences of the testers must be improvised
and creative. Though counter intuitive as the relationship might seem at times, it does work because of both the
technical talent and business processes necessary.
The topic of project management is far to vast and wide to be discussed here. These are the basic distinct phases that
must be considered when planning a penetration test.
- Initiation
- Planning
- Execution
- Control
- Closure
Initiation
There must be a reason for the project. An understanding of what effort and impact tolerance that this will take to
accomplish the project must be in place as well. This is where the initiation stage comes into play.
The initiation stage is about the commitment to the test. There is a definition of the problem and resolution. Also
discussed are the time estimates, financial budgets and impact of the surrounding operations. This is where all
emotional and political support are gathered. A sponsor will step up to fund the idea and take responsibility.
The charter will be signed to launch the project. This charter will include a clear definition of exactly what the
sponsor, (also sometimes known as the customer) wants. There are times that both the sponsor and customer will be the
same person.
Planning
Failure at this point is not an option as the project has begun. There will be no excuses built into the plan. Only
contingencies and risk management options will be accepted. The project definition will be broken down into manageable
and assignable parts. The schedules will be calculated and the assignments will be delegated.
Execution
This is where the plan is in progress. Best practice here is to follow the plan and to make good notes along it's course.
It is almost impossible to accurately document the plan after too many things have happened. Remember, not all penetration
tests result in a breech. The report is what the customer will use to determine if the efforts they paid for were provided
or not.
Control
Being able to make adjustments that are well informed and not counterproductive to other members of the team is what
control is all about. It is about handling the unexpected, the unknowns in such a manner that they are barely noticeable.
This phase develops and maintains the confidence of both the sponsor and client. It lets them know what is going on and
keeps the project on track.
There will be the inevitable status report requests. An answer of "I don't know" only causes the client to lose confidence
in the team. The team members must be able to explain at all times what they are doing and why they are doing it.
Detailed documentation should be kept showing what has already been done and will eventually be developed into the final
project report.
Closure
This is where the deliverables, or also known as the exit criteria of the project have been met. A report entitled the
Administrative Close is a part of this phase and will include all formal acceptance of the results and
agreements to support regarding of clearing all tracks of the tests and the payment.
A meeting of the penetration testing team should be held to discuss lessons learned and review the project as a whole.
Initial Documentation
Agreements made in writing and the understanding between the client and the tester should be made and be perfectly clear
before the execution and monitoring phases of the engagement begin. The final outcome of the test depend on this. The
has the potential of being too dangerous to allow any sloppiness in regards to the cooperation that must exist between
the sponsor and the penetration team.
These are the core doucments that must make up the initiation phase of the penetration test.
- Project Scope
- Indemnity Clause
- "Get out of jail free" Card
- Rules of Engagement
- Communications Plan
- Non-Disclosure Agreements
- Exit Criteria
Project Scope
The Project Scope is a document of high-level view defining the test. It provides the start and end dates,
budget estimates and a list of the people that will be participating as part of the team. If any member of the team
change during the engagement, the client should be informed immediately.
The Statement of Work (SOW) and Project Charter are associated with this as well. This document is
the point that marks the official beginning of the engagement and from this point going forward, failing to finish the
task is not an option.
Indemnity Clause
The Indemnity Clause is a legal statement protecting the tester should something go wrong. The client is
completely and fully informed of all risks prior to the start of the penetration test. This document makes the client
responsible for the decision to move forward and releases the penetration team from liability.
"Get out of Jail Free" Card
The person representing the client and is the project sponsor of the penetration test is authorized to pay for the project.
Should the penetration test be unannounced, there is a good chance that the tester could get caught. If this happens, it
is the responsibility of the sponsor to step in and reveal that this exercise was approved and that all responsibility
lies with the decision maker.
Rules of Engagement
This document restrains the testing team, protects the client and makes sure that none of the activities of the test
will exceed the comfort zone of the project sponsor. There is a comprehensive checklist of activities that is negotiated.
This is thorough and complete as to what tools and techniques the team is allowed to use and explicitly not allowed to
use.
Some examples are, social engineering may be allowed over the phone but not directly in person. DoS tools are not allowed
at all, Rootkits, viral code and illicit servers obtained from unknown sources may not be permitted as they can not be
completely controlled or cleaned up by the testing team.
Should the team write scripts during the test, is the script considered proprietary intellectual property of the testers
or does all of the source code need to be disclosed to the client? The rules of engagement is vital to the outcome of
the test and the team must remain within these boundaries if other legal protections are to be held.
Communications Plan
At times a testing team may encounter a vulnerability that requires immediate attention as the risk is so great to the
client that the reporting of this vulnerability can not wait until the end of the project. For this, there must be a
proper procedure in place. At the same time, should the testers affect a service and have to stop what they are doing,
there must be a way the customer can contact the team and ask for a pause. This usually involves providing an explanation
as well.
The communication plan has to fit within the project scope and type of test that has been ordered by the client. If
there is too much communication, it might not only be dangerous but might also diminish the confidence in the testers
themselves. Needed is a skilled liaison working with the team and client. This liaison must exercise good judgment
in this area and must wear all of these hats.
Non-Disclosure Agreements
The results of the test must be tightly controlled. There has to be a method of proving the report that has to be agreed
upon as well as to the proper destruction of any copies of the notes taken along the way.
The client understandably wants the assurance that the information gathered by the penetration testers is considered
sacred and protected. The company providing the testing services wants referrals and positive feedback, yet they can not
always reveal their clients due to the nature of this sensitive information.
The value of the Licensed Penetration Tester (LPT) is a privately held license from the EC-Council. When experience can
not be explicitly discussed, it is helpful when a baseline of knowledge can be communicated and an organization can back
up the endeavor undertaken to achieve the credential.
Exit Criteria
The Exit Criteria contains the report and a follow-up discussion regarding a schedule of future tests. The
project must have a defined end but must establish with the client what the next step should be. The next project might
be a routine repeat of the previous one or it may become a whole new service based on the results of the current test now
being closed.
This closing meeting of the exit criteria is both an end to the current project and a milestone in a larger scope of
things to come. It is a chance to draft a new charter and start all over again.
Writing an Effective Report
Style and Tone
The major part of the exit criteria is the report. It helps the client reach the objective that caused them to order
the test to begin with. It is also a lasting impression clients will have on the work of the testing team and is written
in business language; the wording being straightforward and to the point.
If problems are revealed, make sure the explanation of the issues are dry, empirical and fact-based. Do not insert
personal opinions and of course never be offensive. Refrain from making the report read like a resume reflecting the
work of the team and do not use words like "I" or "we". Make the report a source of information where the objective of
the test being the object, not the testers themselves.
Many ways can be used to arrange the information delivered. The customer might even have a preference and this preference
should be communicated in the project plan. The summary of information should be accurate but at the same time easily
digestible. Make issues found during the test sound manageable. Fear, uncertainty and doubt have no place in the final
report; it should be the exact opposite. The report should make the client feel that any problems discovered are
manageable and state clearly what needs to be addressed.
Should the test be uneventful, realize that gaping holes in the network were not necessarily expected. Sometimes clients
are on top of things with their network security and the test will show this. Free testing services should never be given
away nor should feelings of guilt be allowed for not finding vulnerabilities. The rules of engagement should never be
violated in the frustration over not finding anything to put in the report.
The best opportunity to demonstrate the value of the test is in the proper documentation and justification of each action
during the test. The report should honestly and accurately explain to the client what took place. The ultimate decision
about whether or not the proper effort and diligence was performed lays with the client.
Elements to Include in the Report
Report writing is a task that is a true skill that is harder than most people think and is a medium in which how the
entire effort of the team is judged. It can also make the difference between repeat business or not; whether a client
believes they received the product and service they paid for.
The report should contain the following six elements for it to be effective.
- Executive summary
- Definitions and conventions
- Vulnerability rated on a critical or accepted scoring system
- Index numbers of industry recognized vulnerability databases, if applicable
- Description of the vulnerabilities found or activities conducted while searching for them
- Summary and conclusion
Executive Summary
The Executive Summary is what tells the readers what they are reading about and what the conclusion is. It is
where you acknowledge the audience and thank them for their time. Make this report as compelling as possible as it
will then be read with interest and critical thought.
If the executive summary is not compelling and dull, it will most likely be discarded no matter how good the information
is and nobody will read it.
Definitions and Conventions
Technical jargon should be avoided if at all possible, but at times this is not possible. Most of the time it is better
to just use the term and move on. However, a brief description as to what the term refers to will help the reader better
understand the meaning of the jargon used.
Vulnerability Rated on a Critical or Accepted Scoring System
A metric should be assigned to all discovered vulnerabilities. This helps the consultant and client collaborate on a
set of priorities. Each issue should be assigned a score that can help in this prioritization. This is subjective and
all assessment tools provide a system of their own.
The scoring scale should be explained in the Definitions and Conventions section of the report.
Should you decided on a scale of 1 to 10, tell the client exactly what "1" represents and what "10" represents. You
should also describe exactly what all of the numbers in between represent as well. Keep all definitions straightforward.
Some vulnerability assessment tools have a built in scoring system. Also available are tools called Common
Vulnerability Scoring Systems (CVSS) that take into account several factors relating to true risk. A description
and more information about CVSS can be found on the Common
Vulnerability Scoring System SIG site.
Index Numbers of Industry Recognized Vulnerability Databases, if applicable
The report that the customer receives should be very informative as far about the vulnerabilities, what their causes were,
how to reproduce them if necessary and how to fix them. Should the vulnerability be indexed in one of the industry
databases, provide the index number and link in the report so that the customer can do further research on their own.
If the test was a black box style, focus on what steps were taken to lead to the discovery of the vulnerability. Allow
the customer to choose to research further using your references if they desire to. Make sure the report is written in
"business language."
If the test was a white box style, it is acceptable to use the reports your assessment tools created automatically. Add
value to your report that will service the client specifically. Customize your report to your customer's specific needs
and system. Do not give the customer a stock report leaving them felt that they have spent good money on something they
could have done themselves.
Description of the Vulnerabilities Found or Activities Conducted While Searching for Them
The title of the vulnerability report should read like a newspaper headline, short but an attention getter.
Keep the reader focused on the risk and not on the tester. Emphasize the reason why the vulnerability is important.
Make sure it is described as to how the weakness was found including the tools and actions used. It is important to keep
the flow of the text going from one topic to the other. If there are details and screen-shots, keep these confined to an
appendix. Give your customer the choice to pursue further details or to just take your word for your findings. Continue
this process throughout the whole report.
Be sure that there is enough evidence to be convincing and provide guidance to an experienced administrator in terms of
reproducing the problem. This should be done without creating a complete tutorial of the issue, and it must be done in
business language while having substance at the same time.
Summary and Conclusion
Summary and conclusion reports should not contain personal opinion. It is vital to accurately state the assessments found
to the client. It is unacceptable to insult, berate or bully the client with statements such as, for example, "This is
a poorly designed network and shows a lack of competency. This is why there were so many vulnerabilities were
discovered." Acceptable reporting should reflect this thought with a more professional, non-opinionated statement, such
as, "The testing team has discovered several vulnerabilities that require immediate attention."
Delivering the Report
These reports are critical and should not be seen or end up in the hands of the wrong people, therefore special
arrangements should be made as to how the report is delivered.
At times a document that is password protected is acceptable, like a password protected PDF (.pdf) document, however,
the passwords on these type of documents are not that difficult to crack. Should a PDF document be used, it is important
to state how long the testing team will bear the responsibility of the security of these documents. The customer should
be advised that the team will keep the document for a set period of time and then the document will be destroyed. This
report may be all that is left of a penetration test that cost the customer thousands of dollars.
Another option is to make a single printout of the report in the customer's building and hand-deliver the report directly
to the project sponsor. A debriefing may be asked so that an explanation of the report can be given. If a PowerPoint is
created for this meeting, both the document and PowerPoint should be properly handled and destroyed.