CAIRIS

Ben

fig:BenPortrait

Image source

Narrative

Activities

Ben is a CREST Registered Penetration Tester at TeamRed LLP; he has been in his current post since finishing his undergraduate degree in computing 4 years ago. He is currently working in a team of three to carry out an infrastructure test for a regional supermarket chain; this also entails evaluating one of their mobile web apps.

Ben spends several times a week speaking with an IT team contact to explain on-going progress. Some of this time is spent clarifying whether certain systems are within scope, and ascertaining whether certain findings of interest are a cause for concern or not. Ben’s clients know their systems better than him, and will [justifiably] challenge concerns he raises if they don’t think assets are that important. On the other hand, Ben also knows that uneducated clients downplay risk, and blasé clients are often slow or reluctant to make any required changes.

As the project draws, to a close, Ben will consider how best to explain fixes for any problems that are found, but IT teams don’t always like being told what they should do. His experience is that teams distrust his unstructured approach to getting at testing targets and, as a result, is sometimes treated with suspicion and occasional passive aggressiveness - particularly if any systems fall over during testing, irrespective of whether this has anything to do with testing.

Attitudes

Ben feels that ethics is an important soft skill. Ben thinks that TeamRed’s code of conduct is largely common sense, but he also knows that unethical behaviour can damage his career should he lose his CREST credentials as a result of an audit by CREST or CESG. Ben realises that any social engineering activities he undertakes can lead to ethical risks given that he is expected to do whatever he has to do to compromise a target; this can include impersonating other people.

Ben finds that working on a testing team can be difficult when the client has commissioned the test because they have to as box-ticking exercise, rather than because they want to. Such tests are particularly difficult when systems are new and the level of accreditation isn’t clear, or the clients are focused on safeguarding proprietary information above all else. Such clients can also complain because they don’t understand what a penetration test entails, and what everyone’s responsibilities are before, during, and after a penetration test. This can be frustrating to Ben, particularly when his client contact is someone who is not security-savvy, or lacks the required authority to action any recommendations made.

While open hostility is rare, Ben often finds his team’s relationship with client IT ‘blue’ team can be frosty at times. During engagements, Ben is proactive in making sure that misunderstandings do not occur due to communication problems. For example, Ben speaks regularly to the IT teams to confirm any expected behaviour during their engagement. While the client’s IT team may not be happy about his team’s presence, he does expect clients to remain professional during testing; this includes ensuring as few people as possible know about any ‘red team’ tests that might be underway, and ensuring that any systems being tested are not modified or made more secure during the testing process.

Aptitudes

Ben takes his professional obligations seriously. He is sometimes asked to help with some specific management research or collaborate on white papers. As a result, he has some experience researching attacks from the literature, or incident reports raised by colleagues. He keeps track of various news feeds that report new vulnerabilities or threats, and attends internal seminars given by his colleagues on different topics of interest. Together, this research and general awareness of attacks and incidents in the wider community helps Ben understand the perspectives taken by attackers. Where the opportunity avails itself, he shares this perspective when keeping his clients appraised of anything new that might impact a current engagement.

Motivations

Ben enjoys looking at problems from the perspective of the black hat. He realises that successful attacks are the product of multiple contributing factors, and is intrigued about the different vectors taken to compromise a target system. As Ben’s experience as grown, the more his security senses have become honed.

Because there is no ‘pen testers guide to ethics’, Ben is motivated by the need to stay within the law. The ease within which laws can be broken is an ever present factor, as are the consequences of a criminal record to his career. For this reason, Ben avoids undertaking any testing which is clearly out of scope or in breach of the Computer Misuse Act, and will double check any technical information provided by clients, such as IP address ranges.

Ben spent time shadowing more senior penetration testers during his initial engagements, and watching how they deal with clients during meetings and conference calls. Now his confidence has now grown, Ben has recently became a mentor for a more junior colleague shadowing him on his current project.

Skills

By learning from incidents and open source reports about attacks and techniques used, Ben attempts to keep up-to-speed with the tools and technologies of black hats. Ben is not, however, a black hat himself and is conscious of the damage to the pen testing community that might arise if people thought he was. As a result, while Ben is free to choose the testing tools he wants, he generally relies on a toolkit of tried and tested tools, and will evaluate the output of any potential new tool before using it on an actual engagement. He makes notes on the usefulness and applicability of any tools he uses and/or customises.

Argumentation Models

Activities

fig:BenActivities

Attitudes

fig:BenAttitudes

Aptitudes

fig:BenAptitudes

Motivations

fig:BenMotivations

Skills

fig:BenSkills

References

Document References

Reference Document Excerpt
Attacker perspective hacker mindset GT concept Acting like an attacker when testing a site.
Blame red team / blue team conflict GT concept Always blamed if what being evaluated goes down.
Black hat techniques inappropriate tool selection GT concept Black hat techniques not always the right tools when doing assessments.
Blase slowness client indifference GT concept Blase clients slow to make changes.
Care about security responsibility to practice GT concept Cares about security.
Client challenge issue issue context GT concept Clients challenge issues if they don’t think its serious.
Cognisant of law legal instincts GT concept Team members cognisant of the law even if they don’t know it to the letter.
Common sense codes responsibility to practice GT concept Codes are common sense rather than useful.
Complaints due to misapprehension test authority GT concept Complaints arise due to misapprehension on what tests entail.
Contributing breach factors hidden risk instincts GT concept Usually multiple contributing factors to a breach.
Crime ends career legal instincts GT concept Getting a criminal record ends your career.
Customer might be right issue context GT concept Sometimes customer is right about issues that might never be realised.
Developer hostility red team / blue team conflict GT concept Developers get hostile when their code ‘baby’ is reviewed.
Do whatever needed red teaming GT concept Will do whatever needs to be done to get at the target.
Doing things because they should client indifference GT concept Some organisations do things because they feel they should rather than because they see value from them.
Double check provided info scoping GT concept Double check ownership of information like IP addresses before test.
Easy to breach CMA scoping GT concept Scoping is important as breaching the CMA is easy.
Emergent engagement reshaping service comprehension GT concept Engagements are sometime reshaped as clients become more knowledgeable.
Ethics is a soft-skill ethics training GT concept Ethics training is a consultancy soft-skill
Ethics passed-on knowledge ethics training GT concept Ethics know-how is passed-on knowledge
Expected behaviour calls informational protocols GT concept Clients occasionally called to confirm expected behaviour.
Expected behaviour coded responsibility to practice GT concept Company’s code of conduct sets out expected behaviour.
Faith sustains industry responsibility to practice GT concept Pen testing industry dies if people lose faith in what we do.
Fear of credentials loss career sensitivity GT concept Being stripped of CREST credentials is a massive hit
Hard to gauge new system accreditation expectation management GT concept Different to gauge level of accreditation expectation on new systems
Head in sand client indifference GT concept Differences of opinion occur when client has their head in the sand
Intuition development shadowing GT concept Juniors develop an intuition for vulnerabilities the more engagements they do.
Job induced caution career sensitivity GT concept Pen testers are cautious because they don’t want to lose their jobs.
Juniors become seniors shadowing GT concept Juniors become seniors after a few engagements.
Juniors learn client management shadowing GT concept Juniors learn how seniors deal with clients during meeting and conference calls.
Juniors with seniors shadowing GT concept Junior tests always work with seniors.
Lack of management authority/buy-in test authority GT concept Some managers may lack buy-in or authority to make changes.
Lack of management skills test authority GT concept Some managers not skills enough to work across organisational boundaries.
Lawyers focus on the proprietary expectation management GT concept Lawyers more concerned about proprietary information than tests.
Literature / blue team research state of the art GT concept Carries out background research on attacks from literature, or associated blue teams.
Management research and white papers state of the art GT concept Asked to conduct management research or write whitepapers.
Methodology audit responsibility to practice GT concept Methodologies reviewed by CESG and CREST.
Methodology explained risk articulation GT concept Methodology followed always explained.
News feeds technical training GT concept Maintains feed from different sources for latest news.
No ethical guides ethics training GT concept No guides or literature on tackling ethical issues
No out-of-scope work legal instincts GT concept Won’t do any work outside the scope of the engagement letter.
No rogues hacker mindset GT concept Selection process emphasised that no rogue hiring.
No singling out risk articulation GT concept Targeted individuals shouldn’t be singled out.
Open source based perspective state of the art GT concept Adversory perspective from open source reports of attacks and techniques/tools used.
Perspective from incidents and forensics state of the art GT concept Adversory perspective gleaned from response to past incidents, and network forensics.
Posture change warning informational protocols GT concept Clients warned if security posture changes during test.
Potential for reputation ruin career sensitivity GT concept Competent enough bad pen testers can ruin reputations
Public domain tool evaluation tool selection GT concept Public domain tools and techniques evaluated before use.
Red team communication expectations informational protocols GT concept Clients told to tell as few people are possible about red teams.
Red team impersonations social engineering GT concept Impersonates other people to get to red team target.
Red/blue team game red team / blue team conflict GT concept Try to avoid red team test turning into a red team - blue team game.
Regulatory box-ticking client indifference GT concept Some customers see pen testing as a regulatory box-ticking exercise.
Replicate seniors shadowing GT concept Juniors watch, keep notes, and try to replicate seniors.
Report unread client indifference GT concept Some companies don’t even read the final report.
Risk sense hidden risk instincts GT concept Sense of risk develops are experience grows
Seniors watch over juniors shadowing GT concept Seniors explain what they do and keep juniors in check.
Service implication understanding service comprehension GT concept Clients should understand implications of the service they are buying.
Testing platform freedom tool selection GT concept Free to chose own testing platform.
Told what to do red team / blue team conflict GT concept IT people don’t take kindly to being told what to do.
Tool logs tool selection GT concept Rely on tools producing logs if there is any comeback on testing carried out.
Training via conference and research technical training GT concept Technical training conference from conferences and internal research.
Trusted toolkit tool selection GT concept Rely on an extensive toolkit of trusted tools.
Uneducated downplay risk client indifference GT concept Uneducated clients downplay risks
Unresponsive red team / blue team conflict GT concept Blue teams tend to be unresponsive rather than hostile.
Wide scope of activities red teaming GT concept Wide scope of activities fall under the red team banner.
Work with clients to fix risk articulation GT concept Will work with clients to help fix any problems found.

External Documents

Document Version Authors Date
career sensitivity GT concept 1 Shamal Faily, Claudia Iacob March 2015
client indifference GT concept 1 Shamal Faily, Claudia Iacob March 2015
ethics training GT concept 1 Shamal Faily, Claudia Iacob March 2015
expectation management GT concept 1 Shamal Faily, Claudia Iacob March 2015
hacker mindset GT concept 1 Shamal Faily, Claudia Iacob March 2015
hidden risk instincts GT concept 1 Shamal Faily, Claudia Iacob March 2015
informational protocols GT concept 1 Shamal Faily, Claudia Iacob March 2015
issue context GT concept 1 Shamal Faily, Claudia Iacob March 2015
legal instincts GT concept 1 Shamal Faily, Claudia Iacob March 2015
red team / blue team conflict GT concept 1 Shamal Faily, Claudia Iacob March 2015
red teaming GT concept 1 Shamal Faily, Claudia Iacob March 2015
responsibility to practice GT concept 1 Shamal Faily, Claudia Iacob March 2015
risk articulation GT concept 1 Shamal Faily, Claudia Iacob March 2015
scoping GT concept 1 Shamal Faily, Claudia Iacob March 2015
service comprehension GT concept 1 Shamal Faily, Claudia Iacob March 2015
shadowing GT concept 1 Shamal Faily, Claudia Iacob March 2015
social engineering GT concept 1 Shamal Faily, Claudia Iacob March 2015
state of the art GT concept 1 Shamal Faily, Claudia Iacob March 2015
technical training GT concept 1 Shamal Faily, Claudia Iacob March 2015
tool selection GT concept 1 Shamal Faily, Claudia Iacob March 2015
test authority GT concept 1 Shamal Faily, Claudia Iacob March 2015

Matt

fig:MattPortrait

Image source

Narrative

Activities

Matt is a director of assurance at TeamRed LLP. He started his career in IT around 12 years ago and, after some time working for one of the bigger consultancies, moved into penetration testing. He has been CHECK and CREST accredited but, in recent years, has taken more responsibility for managing the penetration testing arm of his company.

Matt believes that penetration testing can be a great way of educating clients about security. He strongly believes that fixing and limiting the scope of any engagement is a professional obligation, and he has a moral responsibility to report high impact vulnerabilities as quickly as possible. Matt hopes that, by demonstrating this level of professionalism and recommending solid improvements as a result of any engagement, clients will develop their understanding of penetration testing as a service.

Attitudes

Matt believes that his company’s CREST accreditations are a badge of trust, and wearing the badge means strictly adhering to the prescribed code of ethics, upholding high technical and professional standards. Not maintaining these standards not only undermines client confidence but the entire pen testing industry.

To start building client confidence, scoping meetings are planned with a great deal of attention to detail - particularly given the risk of going out of scope and breaching the CMA. Experience has allowed Matt to get a gut feeling of possible risks that might be present in a system at an early stage, so these meetings are used to confirm his understanding, explain potential threats, and communicate the strategy to be taken when undertaking the engagement.

When delimiting scope for legal and ethical issues, Matt finds that site visits are particularly important for red team testers. Red team testing is highly prized because — within reason — testers are given free reign to do whatever they have to do to exploit a target. As such, such visits help testers understand policies that might be in place, understand the limits of what is and isn’t within scope, and raise any obvious security issues they find now. Such visits also provide cues for asking the right questions about any possible ambiguity.

Once an engagement has been concluded then Matt believes responsible clients should apply any recommendations made as soon as possible. While Matt acknowledges their recommendations constitute advice not orders, he also believes that clients fully understand the implications of not following such as advice. On more than one occasion, Matt has seen clients compromised because they didn’t act on recommendations given fast enough.

Aptitudes

As important as processes and tools are, Matt believes that experience is essential when it comes to identifying caveats to testing. Such caveats may need to be added because information is not made available to the team, the vulnerability data available is incomplete, or there is no obvious consensus about what should be within the scope of engagement.

Matt also feels that the ability to appropriate information is key to a penetration test’s success. In large projects, testers need to share information and findings across engagement phases, and any new vulnerabilities that are found should be shared with everyone that needs to know about them.

Motivations

Managing client expectations is a key motivational driver to Matt, so much so that testers should be given coaching in this area. Client relationships should be unbreakable, and clients should feel confident that any data or vulnerabilities collected are safeguarded, and no testing goes ahead until both testers and clients share the same understanding of what the test’s scope is. If, for some reason, these expectations are not managed then a test’s scope can easily start to creep, as evidenced by ‘change requests’ related to scope. Fortunately, as testing progresses, Matt finds that managing these expectations becomes easier. As these expectations stabilise, scope creep becomes less likely, and clients gain a better understanding of what content to expect in the final report, and how to make the engagement work best for them.

Skills

Matt takes the view that ethics should be ‘designed out’ of any penetration testing practices, such that there should no legal or ethical dilemmas to face. To this end, assurance is key in the processes and techniques used by testers. A prescriptive testing methodology is enforced, and deliverables are quality controlled, such that any collected data is sanitised and all recommendations made are defensible. To support this, Matt feels that testing interactions with 3rd parties should be signed off before any engagement begins.

Argumentation Models

Activities

fig:MattActivities

Attitudes

fig:MattAttitudes

Aptitudes

fig:MattAptitudes

Motivations

fig:MattMotivations

Skills

fig:MattSkills

References

Document References

Reference Document Excerpt
3rd party sign-off requirement 3rd party responsibility GT concept Clients deal with 3rd parties as long there is sign-off from the 3rd parties.
Always something hidden risk instincts GT concept Seen so many systems that there is always something to report.
Care about security responsibility to practice GT concept Cares about security.
CEO responsibility remediation responsibility GT concept The CEO is responsible for the legal implications of all his software.
Client and tester scope model scoping GT concept Clients and pen testers have own ideas about what should be within scope.
Client happiness driven client instincts GT concept If the client is happy then you’re on the right lines.
Client-owned vulnerabilities remediation responsibility GT concept Once vulnerabilities reported, dealing with them is left to client.
Client not fast enough remediation responsibility GT concept Clients compromised by reported vulnerability because they didn’t act fast enough.
Client primary responsibility client instincts GT concept Primary responsibility is to client contracted to.
Client relationship unbreakable client instincts GT concept Client relationship is unbreakable.
Client responsibility for 3rd parties remediation responsibility GT concept Clients responsible for 3rd party problems.
Client scope change request scope creep GT concept Clients sometimes want to change scope.
CREST credentials professional credentials GT concept CREST qualifications act as credentials.
Customer data assurances client instincts GT concept Provide assurances about collected customer data.
Do whatever needed red teaming GT concept Will do whatever needs to be done to get at the target.
Duty to report escalation protocols GT concept Duty to report illegal activities.
Easy to breach CMA scoping GT concept Scoping is important as breaching the CMA is easy.
Emergent engagement reshaping service comprehension GT concept Engagements are sometime reshaped as clients become more knowledgeable.
Ethics designed out engagement structure GT concept Engagements structured to remove ethical concerns.
Exhaustive caveat scope caveats GT concept Report vulnerabilities not exhaustive.
Expectations kill scope creep expectation management GT concept Setting client expectations eliminates scope creep.
Expected client content expectation management GT concept Clients have some expectation of what will be in the report.
Factual information defended information management GT concept Factual information is always defended.
Faith sustains industry responsibility to practice GT concept Pen testing industry dies if people lose faith in what we do.
Hold high standards professional credentials GT concept CREST accreditations mean pen testing firms hold selves to a high standard.
Illegal activities reported upwards escalation protocols GT concept Discovery of illegal activities is reported upwards.
Immediacy is ethical responsibility to practice GT concept Ethical to report vulnerabilities immediately.
Information caveat scope caveats GT concept People with information you don’t have might find other issues.
Job over exploration responsibility to practice GT concept Engaged to do a job, not to explore the client’s infrastructure or email.
Know right questions fieldwork GT concept Good pen testers know the right questions to ask to dig beneath the skins
Last resort logs information management GT concept Logs and raw data are provided for inspection as a last resort.
Literature backs up gut feeling hidden risk instincts GT concept Gut feeling for breaking into a system backed up with open source literature.
Methodology audit responsibility to practice GT concept Methodologies reviewed by CESG and CREST.
Methodology explained risk articulation GT concept Methodology followed always explained.
Moral issue obligations escalation protocols GT concept Obliged to report morally controversial issues.
Need to know sharing information management GT concept Vulnerabilities shared on a need to know basis.
New vulnerabilities shared information management GT concept New vulnerabilities applicable to other clients shared with the team.
No legal dilemmas engagement structure GT concept Because of processes in place, legal dilemmas have never been faced.
Pen test scope limited to tech scoping GT concept Pen test scope narrow and technical, unlike red teams which assess security of organisation.
Policies considered not exploited fieldwork GT concept Policies are considered, but not actively exploited.
Prescriptive methodology engagement structure GT concept Methodology steps are quite prescriptive.
Prize matching malicious red teaming GT concept Red team jobs are highly prized because they match real hacking.
Provide advice not orders remediation responsibility GT concept Provide advice to customers, rather than orders.
Ramifications of vulnerabilities risk articulation GT concept Ramifications of vulnerabilities found explained in report.
Red team legal limits red teaming GT concept Red teams limited only by the bounds of law.
Report QA engagement structure GT concept QA process collate inputs into final report.
Risk sense hidden risk instincts GT concept Sense of risk develops are experience grows
Sanitised evidence information management GT concept Evidence from different sources is sanitised before use.
Scope as communication strategy scoping GT concept Scoping communicates pen testing strategy to client and team.
Scope creep happens. scope creep GT concept Scope creep happens.
Scope easily spreads scope creep GT concept Tests in one area can easily spread to out of scope areas.
Scope ethicity responsibility to practice GT concept Strict scope removes ethical issues.
Scoping meetings scoping GT concept Scoping meetings discuss system and likely attack vectors.
Service implication understanding service comprehension GT concept Clients should understand implications of the service they are buying.
Sharing across phases team protocols GT concept Information shared across consultants working on different engagement phases.
Sharing insights team protocols GT concept Share insights and problems with peers.
Site security problems raised fieldwork GT concept General security problems found during site visits are raised.
Sophisticated customer behaviour service comprehension GT concept Informed customers understand purpose of test, and ask sophisticated questions about what they will get out of test.
Strict adherence to ethics codes professional credentials GT concept Strictly adhere to CREST code of ethics.
Takes every opportunity responsibility to practice GT concept Takes every opportunity to find and report issues.
Tests shift mentality service comprehension GT concept Pen tests shift client mentality and improve level of pen test understanding.
Understanding based on asking service comprehension GT concept Customers who ask for a pen test have a good understanding of what they want.
Wild variation of understanding service comprehension GT concept Customer understanding can vary wildly.

External Documents

Document Version Authors Date
3rd party responsibility GT concept 1 Shamal Faily, Claudia Iacob March 2015
client instincts GT concept 1 Shamal Faily, Claudia Iacob March 2015
engagement structure GT concept 1 Shamal Faily, Claudia Iacob March 2015
escalation protocols GT concept 1 Shamal Faily, Claudia Iacob March 2015
expectation management GT concept 1 Shamal Faily, Claudia Iacob March 2015
fieldwork GT concept 1 Shamal Faily, Claudia Iacob March 2015
hidden risk instincts GT concept 1 Shamal Faily, Claudia Iacob March 2015
information management GT concept 1 Shamal Faily, Claudia Iacob March 2015
professional credentials GT concept 1 Shamal Faily, Claudia Iacob March 2015
red teaming GT concept 1 Shamal Faily, Claudia Iacob March 2015
remediation responsibility GT concept 1 Shamal Faily, Claudia Iacob March 2015
responsibility to practice GT concept 1 Shamal Faily, Claudia Iacob March 2015
risk articulation GT concept 1 Shamal Faily, Claudia Iacob March 2015
service comprehension GT concept 1 Shamal Faily, Claudia Iacob March 2015
scope caveats GT concept 1 Shamal Faily, Claudia Iacob March 2015
scope creep GT concept 1 Shamal Faily, Claudia Iacob March 2015
scoping GT concept 1 Shamal Faily, Claudia Iacob March 2015
team protocols GT concept 1 Shamal Faily, Claudia Iacob March 2015

Further details

CAIRIS model