From Quality Assurance to Security Assurance

00:00:03

Speaker 1: Hello and welcome to today’s webcast. My name is Mason and I’m joined by Gary and Azeem, and we will talk about quality assurance and how to get that security assurance that we more and more need into that picture. And Gary Gary Bostwick, he’s president and CEO of Nora Legion, and he got more than 20 years experience holding product marketing, customer success and safe positions as a manager, manager and individual contributor. His experience covers selling and partnering with enterprises globally, and he founded and successfully grow and sold multiple companies in various industries. So welcome, Gary. And we also are pleased to have Aseem Baxi. He’s the CEO and founder of Omaze. And with a history long history of creating start ups. The latest is Webber Maze. His background is really to drive leading edge product technology and the latest company. Weber may help technology companies to find defects in the latest software build in under 24 hours. So he will talk more about that. And I encourage you all to file your questions on the right hand side during the presentation. You can type in questions to our panel here, and that will make it more interesting. I think. So with that, I hand it over to Gary.

00:02:09

Speaker 2: Great. Thank you very much. Well, thanks, everybody, for joining this session. Hopefully you’ll find it educational and helpful. As mentioned, we very much encourage you to post questions in the Q&A section. Also, we will be presenting both a deck and a couple of videos in the presentation. You have all our emails here. Feel free to reach out to us. Email us directly and ask us for some of this content. We’re happy to share the presentation, happy to share the video, or discuss any of this further. So with no further ado, I’ll jump in. I think you can push forward the slide. The topic of today is from quality assurance to security assurance. And the view that we will be presenting is how you can actually expand QA in order to not just help address quality issues with any product or any solution, but also have the ability to add security and application security into that same process. In the last few years, we’ve really seen a seismic shift in the world of application security and in the world of quality assurance that is now enabling us to bring those two worlds together. And one of the key drivers that has really encouraged that activity to happen is the huge shortage in security professionals around the world. If we look at the numbers today, we’re missing about 1 million security professionals in the US and more than 2 million security professionals globally. And by the end of 2022, that number will jump to more than 3 million. As you know, the demands on application security have grown exponentially with unlawful hackers and other people trying to hack applications and trying to steal data, trying to steal money from organizations. And that has increased the demand. While if you look at legacy solutions that have focused on application security, they really haven’t advanced beyond the world of security professionals. That is now changing. And we’re seeing a seismic shift in the last couple of years where solutions around application security, similar to what Norwegian provides, have been much more developer friendly and QA friendly. And that means that they can be deployed as part of the practices that both developers and QA professionals adapt. And that means that you can fully integrate them into the QA process, as you will see today. Sam, could you jump to the next slide, please? Now, when we talk about a defect in a software and whether we look at it as a quality defect or a security defect, if we look at the formal definitions, they are actually very, very similar. So we believe that looking at them as completely separate dichotomies and completely separate issues doesn’t make sense. You can, if you have the right tools and if you have the right solutions in place, actually look both at security and quality vulnerabilities as very similar issues and you can treat them in a very similar way. Obviously, you have to have the right tools in order to be able to do that, because if you tried to have quality assurance people or the quality assurance team address security bugs or security vulnerabilities without having the ability to understand what they are or with having to deal with many, many false positives and trying to decipher which vulnerabilities are real and which vulnerabilities are not real. That would create a huge challenge. But if they have the right tools and they have the right skill set, then there’s no problem in having them address security and quality bugs in the same way. See now. You can jump to that next slide.

00:06:26

Speaker 1: Sorry about that.

00:06:28

Speaker 2: No problem. And when we really look at the different defects, this is from a presentation at an off meetup that happened took place last year.

00:06:38

Speaker 1: Okay.

00:06:39

Speaker 2: You can see that the way we want to adopt and address these defects is make sure that in the modern development environment. So whether you’re doing continuous integration and continuous delivery or whether you’re using microservices or whether you’re using APIs as part of your environment, you want to make sure that you’re testing both for quality assurance and security assurance. First of all, on every sprint, every time you’re releasing code, every time you’re doing your unit tests, you want to make sure that you’re testing for any vulnerabilities that you have. The next thing is.

00:07:19

Speaker 1: Really don’t.

00:07:20

Speaker 2: Think the way we used to think many years ago when we were addressing deployments in a waterfall environment or something like that, in order to be able to really help improve the security and deploy that ounce of prevention as part of your development process so you don’t have to deploy the pound of cure and waste a lot of money on trying to prevent applications, prevent vulnerabilities after they’re already in your code. You have to think out of the box. You have to take new steps and implement new methodologies as part of your development cycle to get better. The next thing is you really have to understand the applications themselves. You need to have application knowledge. So when you are testing the application both for quality and security vulnerabilities, you are knowledgeable and you know what you’re testing and you can’t limit yourself to just the application. You have to test the entire environment and look at both the code that you’re writing, but also any third party code, any microservices and the compiled application in order to make sure that you don’t leave any vulnerabilities out there. And that really integrates into a new methodology of testing, which when I turn it over to a theme you will learn a lot more about and see how we can actually completely integrate quality assurance and security assurance to drive the security of an organization. Um, to the next slide. All right. When we talk about security vulnerabilities, what are those vulnerabilities? And this is just a very short sample. You will see in the demo that a theme shows a bit later that we have a very long list of security vulnerabilities that we can test for. But they range actually it’s more than 7000 different payloads that can be tested as part of the quality assurance process. But you look at different parts of the application and you want to make sure that they’re secure. The one is the ability to authenticate and get authorization into the system. How are people authenticating? Are there any vulnerabilities in that authentication process? And very importantly, something that historically application security security was very weak at. But today we’re seeing very significant improvements and we have many customer examples that show that is the ability to test both non authenticated and authenticated sites. So how do you do authentication as part of the actual automated testing process and make sure that you’re testing every part of the application? They obviously SSL and TLS, the security protocols. We want to make sure that people are using the right protocols and the right levels. The next ones are really the ability to pull data. And we’re looking at either sensitive, get requests. So what is the data that we’re trying to pull and is an unlawful operator trying to get data from our system that they shouldn’t be getting? And the ability to upload data. So it’s a download and the upload data. Can somebody upload information into our system that will really take advantage of vulnerabilities? Error handling in the code, understanding what are mistakes that people are doing. And last but definitely not least, because we find this any time or almost every time we do a vulnerability scan is really the ability to scan response headers because those leaves you very vulnerable if they’re not managed correctly. And so that’s just a very short sample. There is a much, much longer list of vulnerabilities that can be tested. And as you’ll see in the demo, that can be done in an automated way. So that’s just some background on the the reason that we’re talking about quality assurance and security assurance. Now I’ll turn it over to a team who will actually take you through the process and what we’ve done to integrate the neural engine security solutions or application security solutions with WEMA mate’s quality assurance process. So both can be automated in a seamless manner.

00:11:59

Speaker 1: Okay, So building on this central premise, a defect is the defect. What you see is for a system that does like web image system that is focused on doing traditional QA UI testing, load testing, visual testing, mobile app testing, API testing and getting to that defect is the key for the system. What we’re talking about having achieved is seamlessly plugging in a security testing system such as the one from your region straight into the solution, so that from the customer perspective, that’s using the system, it’s just a flick of a button. And the demo that we show is actually going to show how the process has really been simplified so that adding security is no longer a higher select system. Figure it out. It’s just seamless and effortless to add that to your testing, whether it’s in development, whether it’s integration or in any other part of the system. Taking this sort of overall thought process of making security a first level piece that is constantly done is critical. Nobody debates that security is important. The barrier to entry normally is just the sheer amount of effort. It’s not even cost, it’s effort to put security in on a constant basis. So what do people typically do? They will run security maybe once, sometimes once a quarter, and then pay for the consequences because more and more security hacks are occurring and neural fusion system is seamlessly integrated in. You could use it any time. So what does that use it? Any time really means. What we’ve done is outlined over here sort of the entire development process in development, in integration. On the left hand side, most people are moving towards the CI CD environment in that secure environment, what should be done is that the code, the image API user interface and application security is all simultaneously being checked. It should not happen just in staging and reproduction. The problem is it’s difficult to do. And the demo that we’re going to show you shows how it basically just becomes embedded inside the system. Whether you’re in development and you’re just checking in your individual module or you’re in integration and you’re doing a more serious run where one service or one particular area is being done, application security should be done at that point in time, right? I mean, why would you wait until you get to pre-production and have like 100 security defects appear?

00:14:41

Speaker 2: Absolutely so.

00:14:43

Speaker 1: So the thought process is how could we create a system that would allow you to embed your religion’s extremely superb software in and just have it available to you? Right. Obviously, then when you move into preproduction, you can expand what you’re doing and not just to automated tests, including automated application test application security tests. You can do it exploratory testing, you can advance to pen testing as well as continuing to do application security that you’ve been doing all the way through the development process and then in production. Many people think the testing ends and, you know, everybody’s talking about shifts left, but really you need to shift right to production is an area where both continuous quality testing and continuous security testing is something that can be done and can be done pretty easily. When you have an automated system that is already doing it in every other stage in your life cycle. There’s no reason to slow down. You can take representative sets of test cases from a UI API perspective or security perspective and have them constantly running to detect if anything has happened in operations or in production. So I’m going to go into a demo now. I’m going to share my screen and show you how we’ve brought this system and made it really become seamless to do both QA testing and security testing simultaneously. So gather. You can do a little bit of a dance while I get the screen started. Without video.

00:16:26

Speaker 2: Fortunately, we don’t have video, unfortunately, So nobody’s seeing that.

00:16:31

Speaker 1: So you’re secure. You are thinking. So you’re thinking talents come to the fore now?

00:16:38

Speaker 2: Exactly. I will encourage everybody, though, while you’re doing that, we have quite a few participants, so please go ahead, ask questions, get any clarifications. And as I mentioned earlier, you have all our emails. You’ll see it again as we wrap up. So feel free to follow up with us directly with any questions or any clarifications or any help.

00:17:01

Speaker 1: Perfect. So what we have happening over here is the entire process of how S.R becomes part of QA. In stage one, you basically do the traditional feature regression that’s part of QA testing. So for UI and API tests, you run your UI and API tests. Traditionally this would be done with something like a software automation system. That in turn generates the a ha file, which is basically just a series of actions that are happening at the protocol level between your front end and back end typically. Using those HA files, we’re actually feeding it straight into the neural system and that in turn is based off of those HA files that show the interaction of the system, picking out the type of security tests. And you can see, I don’t know that you said 700 sets of tests that can be potentially picked from I tried picking out a list. I ran out of space and this is the most I could put over here on the right hand side of all the various types of tests that they auto generate based on the HA file and note, no lines of code have been written. It’s just working off of the ha. Cool. Right? And then finally, as an output, you get defects. At the end of the day, not a not a report, but an actual defect. And what I’m going to show you now in a little bit more detail is the how we dive in and actually do this from a system level. What is an end user your experience would be in using the system. So what you see over here is there’s a bunch of test cases. These are UI test cases. In this case, there could be UI, there could be API test cases. Right. And what’s the work that you need to do to add application security? You click on that button, that one done. That’s it. That’s the total amount of effort that you need to put in to say, I want security testing done, and then you need to decide whether you want to do it in development. You want to do it in integration, whether you want to do it in staging. But the actual work was that button click. And you see that security testing is now activated for every one of these UI interactions. So here what we’re going to do is to start a regression. This regression is composed of UI testing on two versions of Chrome Firefox. It’s targeted for the staging environment. You fill in your release notes and you basically let the let the regression start. You put in your release notes and you initiate the regression. What you can configure is whether you want to how you want to do your load testing and how you want to do your security testing. And basically you open up the tab that has the security testing configure inside of it and what modules are going to be run, what tests are going to be run. You can scroll through it or you can just take what the suggestion is. The brilliance over here is the machine learning that neural regions put in is actually looking at the HA file that’s going to be generated from the UI testing and selecting the correct set of tests for you. You can control it if you have the knowledge and you want to specifically target certain areas and they have really nice controls. But as a default you can just take what they have and just slide in security testing. It’s really that easy. So you start off the regression here, you see a regression running over on the right hand side. You can see that a new full regression has been started. And basically now the UI testing has started. You click in here is the UI testing going on for UI or API? Both have moved forward. When that finishes, then the security testing commences. You can see all of the security tests, what level they’ve reached, how far they’ve executed. They’ll keep going until they finish and at the end of the regression, all you basically need to do is to go in and look at the defects that have been raised. So on the left hand side over here, the center of the defect donut has the new bugs that have been found in the regression. The video has been put together because 24 hours have elapsed from the time you started it by clicking on the circle. And when you go into select the defects that you have like to have a look at. So here I’m going in and having a look at the bugs that have been found. I have one bug that’s come from security, one that’s come from load testing, one that’s come from a UI testing in this particular case. And I can go in, open up the defect if it was a UI defect, I have a video attached and I just play the video. If it’s a load testing defect and I click on the load test and I actually have data that shows how the load went wrong and why it’s a defect. And in the case of security, it’s even better. You have literally a defect that tells you what the problem is. So now I’m going to go into the defect. You can see what’s the vulnerability that the near Allegiant system discovered, the what the status is. You can assign it. This is the actual bug. So a developer can immediately say, okay, you have an excess vulnerability in this particular system. On this particular method, please go and start working on it. There’s a screenshot that shows from the UI perspective where the API call went through that resulted in it. You can see the request, the get, you can see the response of the piece. And basically this is more than enough information for a developer and a tester to get together and rapidly fix the security vulnerability. You can also go down here and whoops. So you can also go and assign the defect over here on the right hand side, you can take the defect and say, Yeah, I’m interested in it. The security defect is a priority one. I want to fix it. Change the status to valid new and it immediately gets synchronized into your system. So basically, the goal that we had when we partnered with Neural region was to create a system that seamlessly added security testing and made it just part of the way of life that you have in development rather than an add on that you’re doing at the end of a regression. GUI. Any word you want to add on?

00:23:31

Speaker 2: Nothing to the demo, but we do have a couple of questions that have come in and I’m happy to address those now because we’re already talking about the solution. So maybe we’ll we’ll start with those and actually assume the first question is for both of us. So I’ll start and then I’ll hand it over to you. The question the question was what types of applications can be tested with this solution? So I’ll address the application security part and then you can talk to the QA part for application security. It’s actually a very broad array of applications. It can be web applications, it can be mobile applications from the server side and it can be APIs. One of the great things about the solution being based on on fuzzier technologies that we’re not limited on the API side to HTTP applications. We can test HTTP HTTPS web sockets, JSON, XML, so rest and soap APIs. It’s a very broad array that can be tested. The other part of those web applications is that you can test single page applications even if they have menus that need to be expanded, etc. Those can be expanded because we interact back and forth with the application and don’t just try to crawl it. And also you can test, as I mentioned earlier, authenticated application. So if you’d like as part of that HA file that a theme showed to include authentication or credentials, we can look both at the site in an authenticated way or a non authenticated way. The same applies for API. So it’s a very broad set of applications that can be tested for security. I think you want to address the QA part. Yeah, go ahead. Sorry.

00:25:30

Speaker 1: Absolutely. Similarly, from a QA perspective, and this is one of the reasons why partnering with new religion is so valuable. We address user interfaces on browsers, on mobile platforms, on Windows Native, even. We address APIs on all the formats that got delisted that he does from a security perspective or neuro legion does from a security perspective. And we do load testing in addition. And finally with this composition, we do security testing. So the platforms are basically, if anything, has a user interface. I think X windows, if you have a Unix X Windows system, we haven’t done that yet. But almost anything that you want to test from a user interface or an API, we’re doing set-top box testing for one company where their API and protocol stack is actually quite different, right? Because it’s actually a TCP IP protocol. So it’s a very expansive platform and designed to address a wide variety of platforms and technologies. Equally important, and I think this plays equally well with your Legion, is that you need to do it simultaneously. You need to validate, particularly in UI, that the application works on both all forms of tablets, smartphones and web in many, many applications today. And we’ll do all that simultaneously in the same 24 hour period that we do every other capability that we have. So we have 1000 test cases, two browsers Safari, Mobile, iOS, Android, Windows, Native API will do all of that in that 24 hours. Next question.

00:27:21

Speaker 2: I think that that’s actually a follow on question that just came in. So maybe you want to elaborate on that of how do you provide or how are you able to meet that 24 hour SLA on the entire cycle? And once you answer all, that’s something on the security side.

00:27:37

Speaker 1: It’s great. I mean, the answer is looking for quote unquote tricks, right? The ha file is one of our integration points. We do the same thing with load testing, right? So the ability to add stuff in, looking at it from a pure UI perspective, and if you’re thinking of automation, for example, giving a guaranteed SLA that you will complete the regression of 1000 test cases on two browsers and two mobile platforms is pretty gutsy. We have it baked into our contract. We never miss. The reason we’re able to do that is we have three patents already around using multiple different styles of execution simultaneously and getting to the answer of whether a test case is truly failed or not quickly. So, for example, our system has three different automation systems underneath, two different crowdsourcing systems and multiple different formats of manual testing as well as exploratory testing. So one of the key innovations that we’ve done in the platform is the ability to switch from one type of testing to another. We prep up front that are, for example, in UI, we have test cases, test scripts for automation on multiple automation systems, test scripts for crowdsourcing systems, test scripts for manual for the same test case available.

00:28:54

Speaker 2: Simultaneously.

00:28:55

Speaker 1: So that you can fail over. Even more importantly, when a test case changes, we guarantee that if it’s a modified test case and we got in the invocation, we got some release notes that identified like a ID of what are the changes that are in this particular release. We update the test cases in that same 24 hour period. Gaddy.

00:29:21

Speaker 2: Yeah. I think one of the key things that Aseem showed here was the ability to use a ha file. And that is a very different methodology of actually doing security analysis and application security testing. If you look at most SAS solutions, they rely on a crawler, which means they have to come in and crawl for hours and try to identify the entire application. We’ve flipped that on its head and that’s why the integration with webcomics was so easy, because we can just take and ingest the HA files or the API files that they already have as part of their quality assurance tests and run the security tests on those. And that enables us to fully integrate into the sicced as part of the process. We have customers that are doing 500 deployments a day and the security component now only takes seconds per deploy, and that’s why we’re able to integrate so easily. The other thing, as you saw here today, the solution is fully automated. So we’ve done the integrations. Both systems, both web and neural regions are modern systems. We have APIs that enable us to fully integrate, and you don’t have to switch back and forth between any systems. You don’t have to try to figure out where is the information and what are my workflows. It’s a completely unified workflow that makes it very, very easy to complete these scans. I think there’s another question related to that, actually, which is how do you manage the whole And I think you can address that more broadly around the QA process. How do you manage opening tickets for the developers immediately within their JIRA or Git or whatever they’re using?

00:31:16

Speaker 1: Um, that’s a really good question. So what we’ve done is simplified it to the bug, the defect that I showed and we generate the defect and the act of synchronization has a couple of different points behind it. So first off, if any of you worked for the defect management system, my idea of a terrible way to have a day was to spend an hour in a defect review. You would spend a tremendous amount of time first trying to understand what the defect was. And second, you’d have tons and tons of defects. So the first optimization that we did was no defect goes from our system to a customer system without the customer saying it’s a bug that they want and a priority that they agree with. Right? So by definition, that takes out a tremendous amount of noise. And this is one of the reasons why there’s a defect management system in our portal. Only when a customer says, yes, this is a priority one security defect, I get it. I want to fix it. It’s something new. It’s a priority one. Then it’s automatically synchronized into the customer’s defect management system. It doesn’t matter whether it’s JIRA or the Microsoft Defect management system or the IBM system. It’s all of them are synchronized by our platform. The second part of a defect review that that has been really challenging is I’ll start with a UI one and then I’ll hand it to you for the security point. So from a UI test case perspective or an API test case perspective, I have spent hours and hours in reviews where we’d see one line of text and someone would be like, Oops, the system broke, right? If you’re really, really lucky, you have a defect that says, I went into this area, I clicked on this, I clicked on this screen, clicked on this button, and nothing happened really nicely. The optimization that we did in UI was we actually have a 22nd piece of machine learning based cut footage, 20 seconds of footage that shows you the setup for the bug and the bug actually happening. So at that point in time, the time to review a defect drop to 20 seconds, you watch the video and if you have people in the room that know the system, they can instantly tell you, Oh shit, that’s bad. That’s a priority one, right? Or No, I really don’t care. That’s not a bug. Whereas most of the time in defect reviews, meetings, people that spent time reading a bug, staring off into space, trying to remember the user interface, or trying to actually go into the user interface and replicate it. Oftentimes things I can’t replicate it, right? The video takes care of all of that. So that was the innovation of the UI side. We do the same thing on the API side and on the security side, I think it’s even better because what the newer Legion system gives is a direct trace. Gowdy, over to you.

00:34:06

Speaker 2: Yeah. Perfect. We just got a follow on question that I will roll right into, which is great. But I’m the one of the critical things that we’ve done is both automate the ability to open those vulnerabilities. And as part of the integration with Web, we actually automatically open tickets within the web ticketing system, and those flow directly into the same process that a theme just described. So there’s no no manual intervention. But related to that, and this is the question that came in of how do you deal with. Yep, absolutely.

00:34:45

Speaker 1: Sorry. I want to add one more piece of information to that. So the third part of the defect management cycle that we’ve optimized is once the defect is logged, it’s not reported again. Secondly, once the defect is fixed in the customer system, it is automatically retested by us. Those both are incredibly important because if it’s reported, again, you’re just wasting everybody’s time. And if you don’t retest it when it gets fixed, well, what were you doing right? So really, really the third stage of the defect management process. Back to you guys.

00:35:20

Speaker 2: So I’ll add a fourth stage to that, which is a very common stage and a very big problem historically with with application security solutions, which is false positives. If you’re looking at legacy static analysis solutions, they can have 60, 70% rates of false positives, which is really created huge alert fatigue in the industry. And what we’ve done is we’ve taken a very different approach towards reporting vulnerabilities, and that is we validate that every vulnerability can actually be exposed before we report it. So we might report less vulnerabilities, but that means that we have validated everything that can be exploited and validated, that people can actually impact it. So if we see that there is a vulnerability related to a database, but there’s absolutely no database anywhere in this environment or in this part of the code, we will not report it because it cannot be exploited. And that significantly reduces the noise level that people have to deal with and enables both QA and developers to know that anything that’s reported to them is a real vulnerability. As the theme showed in the demo, they get the screenshot, they get the video, they get the headers, they get the pull request, they get all the information they need in order to take action immediately.

00:36:47

Speaker 1: And I’ll echo that sentiment. One of the banes of automation testing and testing in general is the same. False positives occur for different reasons automation, failure, timeouts, occurring locator ID changes, feature changes, etc. No test cases reported as as a false fail from our system. It’s either a pass or a fail, and part of the patents that we have are around guaranteeing that end outcome that you have a definitive outcome that has a guaranteed level of quality, that it has indeed passed or has indeed failed, so that you don’t have to dig through the false positives and understand which ones are relevant, which ones are higher. Same is true with the suggestion of priority. Again, another great reason why there’s such strong synergies between the two systems.

00:37:38

Speaker 2: Perfect. And that’s all the questions we have right now. So if you want to push forward with the presentation, I think that would be great.

00:37:47

Speaker 1: Okay. So I’ll talk a little bit about Web mates. We’ve been around since October 2015. We have offices now in the US, Netherlands and India. I mentioned earlier that we have three patents granted. Actually, two more are filed of the six and four are in process of getting written. We have a trademark. We’ve been recognized. I think the biggest recognition has been lately. We were accepted by the federal government in the form of F works, which is an Air Force accelerator for phase one. Esper, which is basically they said we have cool technology and we’re starting to move our systems into the Air Force and other DOD networks. So a little bit of background about us from a customer’s perspective. We have customers ranging from TRW, which is a startup, a well funded startup with 20. But with tea with a large amount of money in the kitty. But we started working with them when they had literally no code that actually integrated together. So forget about their MVP, their first builder wasn’t there, and we were already helping them with their testing by being ready with the setup and adding test cases as they’ve been adding capabilities. We’re also helping them generate reports that are being used for FDA approval. They’re building a biomedical device that is used to cryogenically store women’s eggs. It’s a hardware robot with a software cloud system and application. At the other end of the scale, you have like online sites like Priceline that are fairly well known, pushing to production three or four times a day. Many, many different businesses. Rackspace. Hemlock Semiconductor is a Dow Corning company where we’re doing SAP. SAP testing for a wide variety of capabilities that they have in their manufacturing plants. So the applicability of the solution is really broad. Basically, if you have a piece of software and you’re bringing it to market and it has either a mobile web API, we can help you and we can help you really quickly with extremely low amounts of effort needed on your side. We actually call it effortless testing and putting in New Allegiance piece. We now have effortless security. Back to.

00:40:24

Speaker 2: Perfect. And I see a few more questions coming in. So after I address the next couple of slides, we’ll will open it up. So if you have other questions, feel free to post them and we’ll talk about them in a second. So just a bit about Norwegian. The company has been around for a couple of years now and our focus is on application security testing. We provide both a dynamic analysis, security testing solution and an AI driven buzzer, which means we can test for known vulnerabilities. And as mentioned earlier, we today have the broadest set of known vulnerabilities with more than 7000 different payloads that can be sent or utilized. And we can test for unknown vulnerabilities and identify zero days. And we have some very interesting stories from different banks and stock exchanges and government entities that we’ve deployed with and identified some critical zero days that were not identified before because of some of the capabilities that I mentioned earlier. The other thing is that you can do a full CI CD integration and automate the entire process, especially if you want to automate it with QA. But beside that you can automate the entire process of security assurance or application security testing and make sure that from your developer so as close to the code as possible with a dynamic analysis solution through compliance. And I see a couple of questions about that coming in, how we can address every aspect of the organization, including testing for business logic vulnerabilities, which is a very unique capability based on our deep AI capabilities that we’ve developed. I talked about the really broad scan surface, so looking at web applications, mobile applications and various APIs, but making sure that you can run those scans very quickly, very efficiently and with zero false positives and enable developers and QA people to take action very quickly. As you saw in the demo that theme presented, we not only report what the vulnerability is, but we also give all the information about IT and remediation guidelines so those vulnerabilities can be fixed in real time. That’s very, very critical. And this is not a theoretical theme. If you go to the next slide, you can see that there are many customers, tens of customers that are actually using the solution across multiple industries. Definitely, leading industry is financials and insurance. About half our customers fall into that category because those customers are using a lot of APIs. Modern technologies around continuous integration, continuous delivery, and they know that they’re vulnerable. But quite a few technology and cybersecurity company, as you can see here. Playtech, which is a gaming company, Varonis, the Misto, which is Palo Alto Networks and a few others, are all using the solutions to test the security of their applications.

00:43:46

Speaker 1: Across the board.

00:43:48

Speaker 2: And we also have a couple of government entities that are using the solution to secure a broad array of government and security related websites. So the solution has if you go to our website, you can see different case studies on how we’ve helped companies reduce the effort of manual testing or penetration testing and reduce the cost of those by more than 70%, which is huge savings for some organizations. And so that’s a bit about us. As soon as you go to the next slide, maybe we can open it up to a few of the other questions that we’ve received. So I’ll jump in. I’ll address the first one on the security side. You can talk about the QA side, but can I generate a compliance report using your system? So today we have different compliance reports that we can provide. Those reports are they show the entire list of vulnerabilities that you’ve identified. What are all the tests that we’ve run, which tests have passed, which tests have failed? And we’re using various techniques such as OWASP to do all of that work. So, yes, you can generate compliance reports if people are looking specifically for things like GDPR compliance or PCI compliance. We have partners that can use our solution in order to provide those reports because they’re certified providers. But the tool itself can test for all the relevant vulnerability things. You want to take that on the side.

00:45:37

Speaker 1: Yeah, I actually mentioned one of the customers that allows us to talk about it. FDA approval with DMR is exactly that, a compliance report. We provide multiple different parts of compliance reports and we are also PCI compliant ourselves. So yes, we can very definitely provide compliance from the perspective of QA and now we can happily do it for security too.

00:46:07

Speaker 2: Okay, perfect. Good. Any other.

00:46:11

Speaker 1: Questions?

00:46:14

Speaker 2: That’s all I’ve seen posted so far.

00:46:17

Speaker 1: Yeah. The false positives is a repeat. Do you want to revisit that?

00:46:22

Speaker 2: Um, sure. So yeah, it’s the same. Hopefully it’s not the same person said. It’s just somebody who missed it because we have had some people join and and jump on and off. So we got the question about how do we make sure that we reduce false positives again and our process is very different than what legacy application and security solutions have done. We make sure that before we report any vulnerability on the security side, we validate that it can be exploited and it can be exposed to before it’s reported. So we don’t actually crash the system or do anything like that. We just make sure that it can be exposed and then we report it in order to reduce alert fatigue.

00:47:12

Speaker 1: And I’ll repeat what I said. The from a false positive perspective, when we do the execution, the test case results that are all auditable, every single test case, every single outcome, the final outcome is available in the form of an execution audit log. And every single execution is classified. We use the terms true path and true fail versus false fail for false positives. So a true path means that the test case has definitively passed. There is no debate about it. There’s no additional investigation necessary. And a true fail is that the test case has definitively failed. And by the time we finish our execution, whether it’s in 24 hours or it’s in 8 hours for our integration build service, the outcome is guaranteed to be a true pass or a true fail for every single test case that’s there, including if the test case has been modified, because then the target changes, right? So if the test case is actually modified, the from an automation perspective, the test case has legitimately failed because it was written with a previous test case definition and then you modify the feature. However, from a customer perspective, you don’t really care, right? You’re interested. I gave you a new build. Yes, we modified the features. Tell me if the new build works with the newly modified feature. So what we actually do is take that failed test case, update it, rerun and rewrite re execute it, and give a final approval on the new feature or the modified test case. Great. Any other questions?

00:49:02

Speaker 2: None that I see. If you want to jump to the next slide, we can. We can bring it home.

00:49:10

Speaker 1: Okay, then.

00:49:13

Speaker 2: Good. I think that the key message we wanted to provide goes back to one of the first slides that I presented and talking about thinking out of the box and not accepting that legacy solutions and old solutions that don’t really work for you anymore. You don’t have to accept them. There are new solutions that you can deploy. There are creative solutions. There are integrated solutions that help you implement quality assurance, security assurance in a modern environment, test.

00:49:45

Speaker 1: All.

00:49:46

Speaker 2: Of the vulnerabilities and all of the defects that you have in a very quick, very efficient and cost efficient way. And if you are concerned about your coverage in QA or security assurance, if you are interested in expanding how you do QA and how you do security assurance or you’re already doing QA, but you think that you might be vulnerable on the security side, there are new solutions. So the challenge, the the old solutions challenge the old guard, make sure that you look into these new solutions. Obviously, reach out to us again. You have our emails here if you’d like the presentation or you’d like to do a deeper dive into the video or learn more about what are all the types of defects and vulnerabilities and how the process works and pricing and all of those fun things. Feel free to reach out. Both Aseem and I can can help you and go from there. The same old funny last words. If you want to jump to the next slide.

00:50:56

Speaker 1: If I I’d like to point out that there is a recording, there will be a recording available from Bright Talk, where you can listen to this session again if you want to. And it to our panel here, please repeat how the audience can access the slides and the demo that that Asim showed.

00:51:25

Speaker 2: Yeah. So you have our emails here both for Azeem and myself. You can see it’s at Norwegian dot com or ACM at Web Mates. Can’t be much easier than that. Feel free to reach out to us and we are happy to share all the information with you in addition to the bright talk recording. Thanks.

00:51:46

Speaker 1: So thank you, all of you on the panel and all of you in the audience. It was a pleasure to host this session and talk very detailed oriented, I think sufficiently detail oriented about quality assurance and security assurance. So thank you all for attending and have. The safe stay now in these times.

00:52:19

Speaker 2: Thank you all for inviting us and for hosting us. It’s been a pleasure.

00:52:24

Speaker 1: Thanks, everybody.

00:52:26

Speaker 2: Thank you. And that concludes our session.

00:52:31

Speaker 1: Bye bye. By.