Okay, good afternoon. My name is Radek Hranincký. I’m currently finishing my PhD at Brno University of Technology in the Czech Republic. And together with my colleagues from different corners of Europe, we made this paper called “What do incident response practitioners need to know? A skillmap for the years ahead”.
So, I think you may all see the rapid headhunting in our profession, both law enforcement and the private sector are looking for professionals more and more intensively. For instance, there was a big jump in cybersecurity job postings between 2007 to 2013. It was a jump off 74%, which was double the rate of all other IT jobs. Even bigger jump was between 2013-2019, it was a 97% jump. So this is a worldwide problem everywhere are looking for professionals.
These are some statements from the representatives of different countries, for instance USA: 82% of employers report a shortage of cybersecurity skills and 71% believe that this talent gap causes a direct and measurable damage to their organizations. Yeah, similar situation in the UK, in France, in Germany, a shortage of IT security specialists. Italy also has a vast problem in relation to cybersecurity education.
So what are the issues here? We studied some papers and existing surveys, and what do we see? There is a major issue, which is the lack of adequate skills and training and also certifications standards and guidelines. Other studies mentioned here, they revealed that the academic curriculum have a limited amount of security, cybersecurity courses, the offering is very limited.
Also there is a poor alignment between what the education covers and what the market actually demands. If there are some courses they are mostly theory based, and there is insufficient focus on practical exercises.
So we decided to improve that, this is our roadmap. Black is what we did and blue is what we are currently doing or what we are planning to do. So what we did: we identified the essential digital forensics incident response skills based on three inputs.
First input was an expert survey. So we wanted to know what do experts think is needed. Second input, we analyzed existing training courses to see what they cover, what areas, and the third input was job listings. We wanted to see what the market actually demands, what the employers want from the employees.
Second step, we based on these three inputs, we made a map of skills and this map should answer what domains of skills are the most crucial, the most essential, and perhaps serve as a baseline and provide some tips and hints for creating new training courses or updating existing ones.
Currently, this is step number three, we are working on a training course, which should reflect these findings. It will be a pilot course which consists of a theoretical part and of course hands-on labs. Next we want to share these materials with public and make a test run of this course, which will be either a semester course or one week intense seminar.
So first input was the survey of the practitioners. The survey had 15 questions, some are open-ended, some are multi-choice, and there are also, yes/no answers. And we send it to our contacts around Europe to 40 practitioners in countries like Germany, Austria, Ireland, Netherlands, Czech Republic, Lichtenstein and others.
And we received 32 answers. About a third was from the law enforcement or other government agencies, the rest was private sector, and you can see there are different industries: high-tech industry, forensic consulting, but also engineering, financial sector, healthcare, even retail.
This is some demographic information. Here we can see that most of the respondents were from institutions, which had over 500 employees. Also most of them have multiple years of experience in digital forensic incident response, about 30% had over 10 years of experience. So we consider their answers relevant to our paper.
I would point out to one of the questions we asked the practitioners: how did you get your experience, your knowledge, and how you stay up to date? About half addressed their academic qualifications, they mentioned they have some qualifications from academia in fields related to cybersecurity and forensics.
88% also addressed non-academic training when more than a half was a vendor-neutral training by SANS Institute and others. 63% mentioned they are learning from a conference papers, journal papers, and so on. 22% learning on the job, 16% mentioned they gain experience by playing capture the flag competitions, they have different hackathons and other activities, and 16% interaction with the community.
We asked them about what skills they consider the most essential. We focused mostly on hard skills, but we also asked about soft skills. From hard skills, number one was the knowledge of software, but also scripting, yeah, which a bit surprised us that scripting is considered very important here in this domain. 40%, second place, knowledge of computer networks. So networking third place, ability to use general forensic tools. From soft skills it was communication, and critical thinking, and attention to detail.
And we wanted to know what tools, what particular tools they use. And we identified 99 different tools in the answers. The top one was the Autopsy, second place EnCase, but also various Linux utilities as well as self built tools and custom written scripts. So scripting is necessary here as we see. 9% Splunk, Wireshark, Volatility Framework, but there are other tools identified.
We also asked about what challenges they face in investigations and the number one answer here, the number one challenge, is encrypted data, which sort of correlates with the recent surveys that we studied from forensic focus and other sources. So encrypted data is currently the considered it’s the biggest challenge in this domain. Number two, insufficient resources, number three, cloud computing, followed by big data problem legislation and so on.
Then we analyzed the existing training courses, so we use the following methodology: so first we made a keyword search on keywords like “digital forensics”, “incident response” and so on. And we found 42 courses. Those we filtered to the results that were legitimate and that provided sufficient level of detail to allow us to extract information, which reduced the set to 37 courses, 17 were academic, 20 from private sector, from institution like SANS, IACIS, Udemy and so on.
Those we analyzed manually, so we reviewed the topics from the courses, the descriptions, the curricula, and so on and identified the most frequently taught skills. What was interesting that only few courses they explicitly advertise themselves as incident response. Majority of the courses was like part of some broader programs. We also distinguished between the academia and the private sector.
In academia the focus on incident response is more frequent in Master’s programs than Bachelor’s. Most programs, they do not have a dedicated course or incident response, but it’s rather part of some other class, but many universities at least offer one to two courses, which are related to digital forensics.
In private sector, what was very interesting that the majority of the trainings required some, they have some prerequisites: they required essential knowledge from computer science, security and so on. And this knowledge, it has to be acquired beforehand, before the training. Typical duration of these trainings is about to from two to six days with an exception, there was some two week course, but mostly between two to six days.
From the skills most frequent was skills were these: number one investigation techniques, like the forensic process methodology, different steps and so on. Second place: network forensics, followed by operating system forensics and data acquisition. What was interesting that legal issues, which are also important in this profession, and as well as ethical aspects or standards, they were often not part of the courses and new topics like Cloud Forensics or IoT forensics, those were very, very rare.
Next we reviewed the job listings. So again, this is the methodology: first we made again a keyword based search. We made it in January this year, the search revealed from 200 to 22,000 jobs worldwide depending on what keywords we use.
So we downloaded several listings per region. Regions were European states, member states of the EU, Switzerland but also part of the USA. And we deleted non-English listings. Second, we made a dataset from these, we removed listings also that were too general or that were not related to the domain, which resulted in 66 job listings. Those, again, we analyzed manually and evaluated the most required skills.
Our findings: we found out that the most jobs were located in major cities while some offered the option to work remotely, at least partially. Many listings were from well-known companies like Facebook, Amazon, and so on. From education requirements, more than a half required university degree, 38%, Bachelor’s 5% Master’s, 3% PhD. About 44% did not mention specific education requirements, and also especially bigger corporations. 15% required certifications from GIAC, CISSP and other institutions.
But what is important here, that many listings stated that equivalent practical experience is equally valued as the education. So experience is extremely needed here, extremely required, even for entry-level positions. We also made a brief look for internships, and we saw that those offerings are generally very, very low.
From skills, number one, incident handling followed by data analyst, security event and incident logging also frequently required, network forensics, data acquisition and then the rest. What was interesting, this correlates with the survey answers in that majority required to know one or more scripting languages like Python, Bash, PowerShell and others. So scripting is very necessary here. From soft skills, those are mostly overlapping, like logical thinking, being a team player or flexibility, discretion, but also proficiency in English.
So we have these three data sets, three inputs: the survey answers, the courses and job listings, and we want to make a skill map from them. So how to do so? This is our methodology: first, we made a classification of skills. This was necessary because first to unify the terminology, because one thing was called differently in different listings and so on, which would make a chaos in our study, but also there are relations between the topics.
So we made a hierarchical classification tree, which has 132 nodes, those are the skills, divided into four levels, level one to level four, where level one, the root, is digital forensics incident response. Second level has 14 skills, and we were mostly inspired by the ACM Computing Classification System. Other sources we used were mostly scientific papers, existing courses, academic programs, websites, and so on.
Then we made a matrix of skills which the principle is illustrated here. Each row here represents a particular skill: this is L2, L3, L4, this is one skill. And each column stands for one data record. Data record is either survey answer, course description or job listing.
If there is a match for the given skill in the given record there is an X, so X means match, and you can see that a match for a skill also triggers a match for the upper level skill in the same column. So for instance, this match here triggers this, and also this, and also this. Too, this is to preserve the consistency of this.
Next we assessed that, we evaluated the results, calculated the percentage of matches in survey answers in course descriptions, and in job listings. And we also calculated an average match ratio, which we use to estimate the overall importance of each skill.
So let’s look at the results. This is the ranking of the level two skills. So those are those 14 skills on level two, and this is the percentage of appearance: blue in survey answers, red in courses, yellow in job listings, and green is the average of the first three.
So we can see that the first place, most crucial topic here is network forensics, followed by incident handling, operating system forensics, and others. While for the first ones, this is relatively balanced, we can see later it becomes more interesting for instance, investigation techniques.
This is very much covered by courses, but here it is very low for survey and job listings, and we consider that this is because those are mostly general topics, which are considered as automatic by the respondents and employers. Cloud forensics: this is maybe a point for future improvement of courses, because this is seen as a big challenge by survey respondents but is almost never covered by courses.
So let’s dig more deeper. Let’s dig deeper in these topics and see the first three top ones: network forensics. Those are the top level three skills in network forensics, and we can see that number one is encrypted traffic analysis. This is a very, very huge challenge often addressed by survey respondents, but it is very…it is almost not covered by courses, and this is…I think this is a very big gap to fill.
Security event and incident logging: there is high demand by employers, but not so much focus in courses. So maybe another point to improve.
Incident handling: this was the level, the second level two skill category. Well this is interesting because this area was frequently mentioned by both job listings and both survey answers, but we see there are almost no blue bars here. This is because the respondents in the survey, they provided very little info about what sub branches of incident handling are needed the most. The employers (the job listings) are more concrete, so we can see that the incident handling process tools, endpoint attacks, pivoting, domain attacks, and so on are mentioned, but this area seems to be fairly covered by existing courses.
System forensics: this was the number three most essential skill on the level two skills. We can see that there are different systems. Number one is Windows Forensics, which is obvious because Windows systems have about 70% of market share on desktop computers, so this absolutely makes sense. Number two, Linux Forensics. Yes, Linux is a major OS on servers, this also makes sense that this is on the second place.
But what about mobile forensics, mobile device forensics here? We see that it is very much covered by courses, many courses focus on mobile forensics, but there is absolutely zero demand in job listings, so this may be because in the past mobile forensics was, mobile device or something new, and mobile forensics and security was considered as a big challenge as the mobile technologies were growing, but nowadays everybody has a cell phone and the courses pretty much cover it maybe even more than necessary.
So maybe the importance of this area is overrated, and maybe the courses should provide more space to other topics, maybe some newer, like count forensics, IoT forensics. So this is system forensics, and this is the last topic that I have visualized here.
So to conclude this, I hope that our skillmap will help to designing future courses or improving existing ones. Overall, I would say that much of the existing content of the courses is still relevant, but definitely there are gaps to be filled and should be filled.
What is interesting is that there is a significant difference between what do the professionals think it is most crucial, what skills are most required, and between the actual contents of courses. Example for itself is the analysis of encrypted data. It’s considered as a big, big, big challenge, but it has a very little space in existing trainings and courses.
Most employers also want previous experience. This is very important, more important than we thought it would be in the listings. So even for entry level position experience is necessary and it’s very, very highly valued. Some also say it’s almost equally valued as education. So this even emphasizes the importance of hands-on training and the courses should, and we think they should give more space to practical exercises.
About the soft skills, they are also frequently mentioned in our study, but we did not particularly focus on them. Our skillmap was focused on hard skills, but we of course think that the soft skills are equally important. Okay, that’s all from my presentation. Thank you for attention. And are there any questions?