PENGUMUMAN...!!!


SOLAT HAJAT DI TUNDA KE 24HB OGOS 2008 - UST SALLEH MAN

Friday, May 26, 2023

Automating REST Security Part 2: Tool-based Analysis With REST-Attacker

Our previous blog post described the challenges in analyzing REST API implementations. Despite the lack of REST standardization, we learned that similarities between implementations exist and that we can utilize them for tool-based REST security analysis.

This blog post will now look at our own implementation. REST-Attacker is a free software analysis tool specifically built to analyze REST API implementations and their access control measures. Using REST-Attacker as an example, this blog post will discuss how a REST security tool can work and where it can improve or streamline the testing process, especially in terms of automation.

Author

Christoph Heine

Overview

 Premise

REST-Attacker was developed as part of a master's thesis at the Chair for Network & Data Security at the Ruhr University Bochum. The primary motivation behind creating REST-Attacker was to evaluate how far we could push automation for REST security analysis. Hence, REST-Attacker provides several automation features such as automated test generation, test execution, and API communication. The tool essentially takes a "lazy tester" approach that tries to minimize the necessary amount of manual interaction as much as possible.

Creating a test run requires an OpenAPI file describing the REST API. Optional configuration, such as authentication credentials, can be provided to access protected API endpoints or run advanced test cases. Based on the API description and configuration, the tool can automatically generate complete test runs and execute them automatically. For this purpose, the current release version provides 32 built-in security test cases for analyzing various security issues and best practices.

How Testing Works

REST-Attacker can be used as a stand-alone CLI tool or as a Python module for integration in your own toolchain. In this blog post, we will mainly focus on running the tool via CLI. If you want to learn more about advanced usage, we recommend you read the docs.

Starting a basic test run looks like this:

python3 -m rest_attacker openapi.json --generate

openapi.json is an OpenAPI file that describes the API we want to test. The --generate flag activates load-time test generation to automatically create a test run. In practice, this means that the tool passes the OpenAPI file to a test generation function of every available test case, which then returns a list of tests for the specific API. After creating the test run, REST-Attacker executes all tests one by one and saves the results.

There's also a second option for run-time test generation using the --propose flag:

python3 -m rest_attacker openapi.json --generate --propose

In comparison to --generate, which creates tests from the OpenAPI description before starting the test run, --propose generates tests during a test run by considering the results of already executed tests. This option can be useful for some test cases where we want to take the responses of the API into account and run a follow-up test based on the observed behavior.

Both test generation methods can significantly speed up testing because they allow the creation of entire test runs without manual input. However, their feasibility often heavily depends on the verbosity and accuracy of the configuration data. Remember that many definitions, such as security requirements, are optional in the OpenAPI format, i.e., services can choose to omit them. API descriptions can also be outdated or contain errors, particularly if they are unofficial user-created versions. Despite all these limitations, an automated generation often works surprisingly well.

If you don't want to use the tool's generators, test runs can also be specified manually. For this purpose, you just pass a list of tests, including their serialized input parameters, via a config file:

python3 -m rest_attacker openapi.json --run example_run.json

Advanced Automation

So far, we have only covered the automation of the test generation. However, what's even more interesting is that we can also automate much of the test execution process in REST-Attacker. The challenging part here is the streamlining of API communication. If you remember our previous blog post, you know that it basically involves these three steps:

  1. Preparing API request parameters
  2. Preparing access control data (handling authentication/authorization)
  3. Sending the request

Since most REST APIs are HTTP-based, step 3. is relatively trivial as any standard HTTP library will do the job. For example, REST-Attacker uses the popular Python requests module for its request backend. Step 1. is part of the test generation process and can be realized by using information from the machine-readable OpenAPI file, which we've already discussed. In the final step, we have to look at the access control (step 2.), which is especially relevant for security testing. Unfortunately, it is a bit more complex.

The problem is generally not that REST APIs use different access control methods. They are either standardized (HTTP Basic Auth, OAuth2) or extremely simple (API keys). Instead, complications often arise from the API-specific configuration and requirements for how these methods should be used and how credentials are integrated into the API request. For example, implementations may decide:

  • where credentials are located in the HTTP request (e.g., header, query, cookie, ...)
  • how credentials are encoded/formatted (e.g., Base64 encoding or use of keywords)
  • whether a combination of methods is required (e.g., API key + OAuth2)
  • (OAuth2) which authorization flows are supported
  • (OAuth2) which access scopes are supported
  • ...

Thereby, we cannot rely on an access control method, e.g., OAuth2, being used in the same way across different APIs. Furthermore, a lot of this information cannot be described in the OpenAPI format, so we have to find another solution. In REST-Attacker, we solve this problem with an additional custom configuration for access control. An example can be seen below (unfold it):

{     "schemes": {         "scheme0": {             "type": "header",             "key_id": "authorization",             "payload": "token {0}",             "params": {                 "0": {                     "id": "access_token",                     "from": [                         "token0",                     ]                 }             }         }     },     "creds": {         "client0": {             "type": "oauth2_client",             "description": "OAuth Client",             "client_id": "aabbccddeeff123456789",             "client_secret": "abcdef12345678998765431fedcba",             "redirect_uri": "https://localhost:1234/test/",             "authorization_endpoint": "https://example.com/login/oauth/authorize",             "token_endpoint": "https://example.com/login/oauth/token",             "grants": [                 "code",                 "token"             ],             "scopes": [                 "user"             ],             "flags": []         }     },     "required_always": {         "setting0": [             "scheme0"         ]     },     "required_auth": {},     "users": {         "user0": {             "account_id": "user",             "user_id": "userXYZ",             "owned_resources": {},             "allowed_resources": {},             "sessions": {                 "gbrowser": {                     "type": "browser",                     "exec_path": "/usr/bin/chromium",                     "local_port": "1234"                 }             },             "credentials": [                 "client0"             ]         }     } }

The config file contains everything required for getting access to the API. schemes define location and encoding of credentials in the HTTP request, while credentials contain login credentials for either users or OAuth2 clients. There are also definitions for the required access control schemes for general access to the API (required_always) as well as for user-protected access (required_auth). For the purpose of authorization, we can additionally provide user definitions with session information. The latter can be used to create or access an active user session to retrieve OAuth2 tokens from the service.

Starting REST-Attacker with an access control config is similar as before. Instead of only passing the OpenAPI file, we use a folder that contains all configuration files:

python3 -m rest_attacker cfg/example --generate

REST-Attacker completely handles all access control requirements in the background. Manual intervention is sometimes necessary, e.g., when there's a confirmation page for OAuth2 authorization. However, most of the steps, from selecting the proper access control schemes to retrieving OAuth2 tokens and creating the request payload, are all handled by REST-Attacker.

Interpreting Results

After a test run, REST-Attacker exports the test results to a report file. Every report gives a short summary of the test run and the results for each executed test case. Here you can see an example of a report file (unfold it):

{     "type": "report",     "stats": {         "start": "2022-07-16T14-27-20Z",         "end": "2022-07-16T14-27-25Z",         "planned": 1,         "finished": 1,         "skipped": 0,         "aborted": 0,         "errors": 0,         "analytical_checks": 0,         "security_checks": 1     },     "reports": [         {             "check_id": 0,             "test_type": "security",             "test_case": "https.TestHTTPAvailable",             "status": "finished",             "issue": "security_flaw",             "value": {                 "status_code": 200             },             "curl": "curl -X GET http://api.example.com/user",             "config": {                 "request_info": {                     "url": "http://api.example.com",                     "path": "/user",                     "operation": "get",                     "kwargs": {                         "allow_redirects": false                     }                 },                 "auth_info": {                     "scheme_ids": null,                     "scopes": null,                     "policy": "DEFAULT"                 }             }         }     ] }

Individual test reports contain a basic classification of the detected behavior in the issue parameter and the detailed reasons for this interpretation in the value object. The meaning of the classification depends on the test case ID, which is stored in the test_case parameter. In the example above, the https.TestHTTPAvailable checks if an API endpoint is accessible via plain HTTP without transport security (which is generally considered unsafe). The API response is an HTTP message with status code 200, so REST-Attacker classifies the behavior as a flaw.

By default, reports also contain every test's configuration parameters and can be supplied back to the tool as a manual test run configuration. This is very useful if we want to reproduce a run to see if detected issues have been fixed.

python3 -m rest_attacker openapi.json --run report.json

Conclusion

By now, you should know what REST API tools like REST-Attacker are capable of and how they can automate the testing process. In our next and final blog post, we will take a deeper look at practical testing with the REST-Attacker. To do this, we will present security test categories that are well-suited for tool-based analysis and investigate how we can apply them to test several real-world API implementations.

Acknowledgement

The REST-Attacker project was developed as part of a master's thesis at the Chair of Network & Data Security of the Ruhr University Bochum. I would like to thank my supervisors Louis Jannett, Christian Mainka, Vladislav Mladenov, and Jörg Schwenk for their continued support during the development and review of the project.

More information


Posted by at No comments:

Game Store Investing

The feeling of long term store ownership might be described as failing upwards. There are regular setbacks, characterized by the phrase, two steps forward, one step back. The saddest thing ever, I think, would be to look up after ten or twenty years, and see things exactly the same. You are the same. The store is the same. The money is the same. You are the victim of a Buy-A-Job. Top stores, those failing upwards, are run by owners who believe they are heading in a positive direction and they work towards that progress. How do you work towards progress?

First, invest in yourself. Besides paying yourself from day one, which means having a larger startup budget, you want to invest a percentage of your income for retirement. This might seem extravagant, but honestly, when else will you do it? Retirement savings can also be used for buying a home, college, or a number of other investments in yourself. Starting younger means you won't have to scramble when you've finally "made it" in this trade.

Investing in yourself also means learning new skills. This is going to happen anyway when you first start a business, but regular personal development of your skillset is harder over time. I'm currently paying someone for graphic design work and I'm about to hand over my website to an employee. This is mostly because I've lost interest. I'm instead working on learning Spanish and RV repair. My eye is not on the ball.

Second, invest in your store. Avoid purchases that don't add value. My investors describe this as "pimp value," or what today we might call flexing. Buying a van and wrapping it as a billboard, a vintage pinball machine, or yet another rubber statue is pimp value. Now that we've tripled our inventory and seen what true investment can do for us, pimp value is clearly seen as net profit deferral. Avoid pimp value. Flex with profit.

The most solid store investment you can make is more inventory. Even bad inventory that underperforms is better than pimp value. Avoid spending money on Other Peoples Property. We spent $160K expanding our game space to a second floor, just in time for the Magic boom to end. We paid it off during COVID, and event attendance may never fully recover. When we one day move out of our space, our landlord will likely tear down our second floor. It's that idiosyncratic to our operation. 

Third, use systems to invest. Give yourself a salary based on a dynamic number. My salary is based on the low end of the average range of a retail store manager. This lets me increase it over time without feeling like I'm taking too much. This is not all of my take home pay, but it's the baseline I can take, even during hard times. After you have a salary, take a percentage of your income and invest it in a 401K or IRA. The ideal number is supposedly 15-20%. If you can't possibly imagine doing that right now, try 1%. Then increase it slowly. Anything is good.

After you've paid yourself, repeat this process with your business. Take another percentage for operational improvements and inventory increases. This is easy to do during your windfall months, like the summer or after the holidays. A 10% inventory increase has been my guideline for a while. I'm always progressing if my inventory is increasing. This also lets me play with inventory performance metrics as a means to increase profit. It's not just more stuff, it's an engine for profitability and a way to keep me engaged.

Pro Tip: Take 10% of your net profit and put it aside for improvements and repairs. Early on when profits are thin, it might seem all your profit goes to improvements and repairs. Eventually you'll get a handle on entropy. The goal is to find yourself sitting around a table with staff and scratching your heads because you don't know how to improve the store further. You've done all the projects on your white board so you'll need to get creative. This is where you want to be! Watch your store become a leader in the field, rather than worrying about a broken toilet.

These might seem like small things, but that's the point. You sacrifice a small amount here and there for big results later. Finally, don't be afraid to take home your net profit. Remember that the R in ROI is Return. If you aren't taking money out in the form of profit, I think you're doing it wrong. Enjoy your life. Develop hobbies outside of gaming. Take care of your health. Plan a vacation that's not to a trade show. You don't need to take pride in your success, but you certainly don't owe anyone an explanation.

Posted by at No comments:

Saturday, August 6, 2022

Top 14 Free Websites to Learn Hacking this 2018

  1. HackRead: HackRead is a News Platform that centers on InfoSec, Cyber Crime, Privacy, Surveillance, and Hacking News with full-scale reviews on Social Media Platforms.
  2. The Hacker News: The Hacker News — most trusted and widely-acknowledged online cyber security news magazine with in-depth technical coverage for cybersecurity.
  3. Exploit DB: An archive of exploits and vulnerable software by Offensive Security. The site collects exploits from submissions and mailing lists and concentrates them in a single database.
  4. SecTools.Org: List of 75 security tools based on a 2003 vote by hackers.
  5. DEFCON: Information about the largest annual hacker convention in the US, including past speeches, video, archives, and updates on the next upcoming show as well as links and other details.
  6. Black Hat: The Black Hat Briefings have become the biggest and the most important security conference series in the world by sticking to our core value: serving the information security community by delivering timely, actionable security information in a friendly, vendor-neutral environment.
  7. KitPloit: Leading source of Security Tools, Hacking Tools, CyberSecurity and Network Security.
  8. Hakin9: E-magazine offering in-depth looks at both attack and defense techniques and concentrates on difficult technical issues.
  9. Metasploit: Find security issues, verify vulnerability mitigations & manage security assessments with Metasploit. Get the worlds best penetration testing software now.
  10. Hacked Gadgets: A resource for DIY project documentation as well as general gadget and technology news.
  11. Phrack Magazine: Digital hacking magazine.
  12. NFOHump: Offers up-to-date .NFO files and reviews on the latest pirate software releases.
  13. Packet Storm: Information Security Services, News, Files, Tools, Exploits, Advisories and Whitepapers.
  14. SecurityFocus: Provides security information to all members of the security community, from end users, security hobbyists and network administrators to security consultants, IT Managers, CIOs and CSOs.
Posted by at No comments:
Subscribe to: Posts (Atom)