Green Web Hosting! This site hosted by DreamHost.

Wednesday, May 31

Msticpy - Microsoft Threat Intelligence Security Tools

Microsoft Threat Intelligence Python Security Tools.

msticpy is a library for InfoSec investigation and hunting in Jupyter Notebooks. It includes functionality to:

  • query log data from multiple sources
  • enrich the data with Threat Intelligence, geolocations and Azure resource data
  • extract Indicators of Activity (IoA) from logs and unpack encoded data
  • perform sophisticated analysis such as anomalous session detection and time series decomposition
  • visualize data using interactive timelines, process trees and multi-dimensional Morph Charts

It also includes some time-saving notebook tools such as widgets to set query time boundaries, select and display items from lists, and configure the notebook environment.



The msticpy package was initially developed to support Jupyter Notebooks authoring for Azure Sentinel. While Azure Sentinel is still a big focus of our work, we are extending the data query/acquisition components to pull log data from other sources (currently Splunk, Microsoft Defender for Endpoint and Microsoft Graph are supported but we are actively working on support for data from other SIEM platforms). Most of the components can also be used with data from any source. Pandas DataFrames are used as the ubiquitous input and output format of almost all components. There is also a data provider to make it easy to and process data from local CSV files and pickled DataFrames.

The package addresses three central needs for security investigators and hunters:

  • Acquiring and enriching data
  • Analyzing data
  • Visualizing data

We welcome feedback, bug reports, suggestions for new features and contributions.


Installing

For core install:

pip install msticpy

If you are using MSTICPy with Azure Sentinel you should install with the "azsentinel" extra package:

pip install msticpy[azsentinel]

or for the latest dev build

pip install git+https://github.com/microsoft/msticpy


Documentation

Full documentation is at ReadTheDocs

Sample notebooks for many of the modules are in the docs/notebooks folder and accompanying notebooks.

You can also browse through the sample notebooks referenced at the end of this document to see some of the functionality used in context. You can play with some of the package functions in this interactive demo on mybinder.org.


Log Data Acquisition

QueryProvider is an extensible query library targeting Azure Sentinel/Log Analytics, Splunk, OData and other log data sources. It also has special support for Mordor data sets and using local data.

Built-in parameterized queries allow complex queries to be run from a single function call. Add your own queries using a simple YAML schema.

Data Queries Notebook


Data Enrichment

Threat Intelligence providers

The TILookup class can lookup IoCs across multiple TI providers. built-in providers include AlienVault OTX, IBM XForce, VirusTotal and Azure Sentinel.

The input can be a single IoC observable or a pandas DataFrame containing multiple observables. Depending on the provider, you may require an account and an API key. Some providers also enforce throttling (especially for free tiers), which might affect performing bulk lookups.

TIProviders and TILookup Usage Notebook


GeoLocation Data

The GeoIP lookup classes allow you to match the geo-locations of IP addresses using either:

GeoIP Lookup and GeoIP Notebook


Azure Resource Data, Storage and Azure Sentinel API

The AzureData module contains functionality for enriching data regarding Azure host details with additional host details exposed via the Azure API. The AzureSentinel module allows you to query incidents, retrieve detector and hunting queries. AzureBlogStorage lets you read and write data from blob storage.

Azure Resource APIs, Azure Sentinel APIs, Azure Storage


Security Analysis

This subpackage contains several modules helpful for working on security investigations and hunting:


Anomalous Sequence Detection

Detect unusual sequences of events in your Office, Active Directory or other log data. You can extract sessions (e.g. activity initiated by the same account) and identify and visualize unusual sequences of activity. For example, detecting an attacker setting a mail forwarding rule on someone's mailbox.

Anomalous Sessions and Anomalous Sequence Notebook


Time Series Analysis

Time series analysis allows you to identify unusual patterns in your log data taking into account normal seasonal variations (e.g. the regular ebb and flow of events over hours of the day, days of the week, etc.). Using both analysis and visualization highlights unusual traffic flows or event activity for any data set.


Time Series


Visualization

Event Timelines

Display any log events on an interactive timeline. Using the Bokeh Visualization Library the timeline control enables you to visualize one or more event streams, interactively zoom into specific time slots and view event details for plotted events.


Timeline and Timeline Notebook


Process Trees

The process tree functionality has two main components:

  • Process Tree creation - taking a process creation log from a host and building the parent-child relationships between processes in the data set.
  • Process Tree visualization - this takes the processed output displays an interactive process tree using Bokeh plots.

There are a set of utility functions to extract individual and partial trees from the processed data set.


Process Tree and Process Tree Notebook


Data Manipulation and Utility functions

Pivot Functions

Lets you use MSTICPy functionality in an "entity-centric" way. All functions, queries and lookups that relate to a particular entity type (e.g. Host, IpAddress, Url) are collected together as methods of that entity class. So, if you want to do things with an IP address, just load the IpAddress entity and browse its methods.

Pivot Functions and Pivot Functions Notebook


base64unpack

Base64 and archive (gz, zip, tar) extractor. It will try to identify any base64 encoded strings and try decode them. If the result looks like one of the supported archive types it will unpack the contents. The results of each decode/unpack are rechecked for further base64 content and up to a specified depth.

Base64 Decoding and Base64Unpack Notebook


iocextract

Uses regular expressions to look for Indicator of Compromise (IoC) patterns - IP Addresses, URLs, DNS domains, Hashes, file paths. Input can be a single string or a pandas dataframe.

IoC Extraction and IoCExtract Notebook


eventcluster (experimental)

This module is intended to be used to summarize large numbers of events into clusters of different patterns. High volume repeating events can often make it difficult to see unique and interesting items.



This is an unsupervised learning module implemented using SciKit Learn DBScan.

Event Clustering and Event Clustering Notebook


auditdextract

Module to load and decode Linux audit logs. It collapses messages sharing the same message ID into single events, decodes hex-encoded data fields and performs some event-specific formatting and normalization (e.g. for process start events it will re-assemble the process command line arguments into a single string).


syslog_utils

Module to support an investigation of a Linux host with only syslog logging enabled. This includes functions for collating host data, clustering logon events and detecting user sessions containing suspicious activity.


cmd_line

A module to support he detection of known malicious command line activity or suspicious patterns of command line activity.


domain_utils

A module to support investigation of domain names and URLs with functions to validate a domain name and screenshot a URL.


Notebook widgets

These are built from the Jupyter ipywidgets collection and group common functionality useful in InfoSec tasks such as list pickers, query time boundary settings and event display into an easy-to-use format.


 



More Notebooks on Azure Sentinel Notebooks GitHub

Azure Sentinel Notebooks

Example notebooks:

View directly on GitHub or copy and paste the link into nbviewer.org


Notebook examples with saved data

See the following notebooks for more examples of the use of this package in practice:


Supported Platforms and Packages
Contributing

For (brief) developer guidelines, see this wiki article Contributor Guidelines

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.



More information

posted by safer cooler greener at 5:09 PM 0 Comments

inBINcible Writeup - Golang Binary Reversing

This file is an 32bits elf binary, compiled from go language (i guess ... coded by @nibble_ds ;)
The binary has some debugging symbols, which is very helpful to locate the functions and api calls.

GO source functions:
-  main.main
-  main.function.001

If the binary is executed with no params, it prints "Nope!", the bad guy message.

~/ncn$ ./inbincible 
Nope!

Decompiling the main.main function I saw two things:

1. The Argument validation: Only one 16 bytes long argument is needed, otherwise the execution is finished.

2. The key IF, the decision to dexor and print byte by byte the "Nope!" string OR dexor and print "Yeah!"


The incoming channel will determine the final message.


Dexor and print each byte of the "Nope!" message.


This IF, checks 16 times if the go channel reception value is 0x01, in this case the app show the "Yeah!" message.

Go channels are a kind of thread-safe queue, a channel_send is like a push, and channel_receive is like a pop.

If we fake this IF the 16 times, we got the "Yeah!" message:

(gdb) b *0x8049118
(gdb) commands
>set {char *}0xf7edeef3 = 0x01
>c
>end

(gdb) r 1234567890123456
tarting program: /home/sha0/ncn/inbincible 1234567890123456
...
Yeah!


Ok, but the problem is not in main.main, is main.function.001 who must sent the 0x01 via channel.
This function xors byte by byte the input "1234567890123456" with a byte array xor key, and is compared with another byte array.

=> 0x8049456:       xor    %ebp,%ecx
This xor,  encode the argument with a key byte by byte

 The xor key can be dumped from memory but I prefer to use this macro:

(gdb) b *0x8049456
(gdb) commands
>i r  ecx
>c
>end
(gdb) c

 Breakpoint 2, 0x08049456 in main.func ()
ecx            0x12 18

 Breakpoint 2, 0x08049456 in main.func ()
ecx            0x45 69

 Breakpoint 2, 0x08049456 in main.func ()
ecx            0x33 51

 Breakpoint 2, 0x08049456 in main.func ()
ecx            0x87 135

 Breakpoint 2, 0x08049456 in main.func ()
ecx            0x65 101

 Breakpoint 2, 0x08049456 in main.func ()
ecx            0x12 18

 Breakpoint 2, 0x08049456 in main.func ()
ecx            0x45 69

 Breakpoint 2, 0x08049456 in main.func ()
ecx            0x33 51

 Breakpoint 2, 0x08049456 in main.func ()
ecx            0x87 135

 Breakpoint 2, 0x08049456 in main.func ()
ecx            0x65 101

 Breakpoint 2, 0x08049456 in main.func ()
ecx            0x12 18

 Breakpoint 2, 0x08049456 in main.func ()
ecx            0x45 69

 Breakpoint 2, 0x08049456 in main.func ()
ecx            0x33 51

 Breakpoint 2, 0x08049456 in main.func ()
ecx            0x87 135

 Breakpoint 2, 0x08049456 in main.func ()
ecx            0x65 101

 Breakpoint 2, 0x08049456 in main.func ()
ecx            0x12 18

The result of the xor will compared with another array byte,  each byte matched, a 0x01 will be sent.

The cmp of the xored argument byte,
will determine if the channel send 0 or 1


(gdb) b *0x0804946a
(gdb) commands
>i r al
>c
>end

At this point we have the byte array used to xor the argument, and the byte array to be compared with, if we provide an input that xored with the first byte array gets the second byte array, the code will send 0x01 by the channel the 16 times.


Now web have:

xorKey=[0x12,0x45,0x33,0x87,0x65,0x12,0x45,0x33,0x87,0x65,0x12,0x45,0x33,0x87,0x65,0x12]

mustGive=[0x55,0x75,0x44,0xb6,0x0b,0x33,0x06,0x03,0xe9,0x02,0x60,0x71,0x47,0xb2,0x44,0x33]


Xor is reversible, then we can get the input needed to dexor to the expected values in order to send 0x1 bytes through the go channel.

>>> x=''
>>> for i in range(len(xorKey)):
...     x+= chr(xorKey[i] ^ mustGive[i])
... 
>>> print x

 G0w1n!C0ngr4t5!!


And that's the key :) let's try it:

~/ncn$ ./inbincible 'G0w1n!C0ngr4t5!!'
Yeah!

Got it!! thanx @nibble_ds for this funny crackme, programmed in the great go language. I'm also a golang lover.


Read more


posted by safer cooler greener at 8:41 AM 0 Comments

Tuesday, May 30

Top 15 Best Operating System Professional Hackers Use

Top 10 Best Operating System Professional Hackers Use

Top 15 Best Operating System Professional Hackers Use

Top 15 Best Operating System Professional Hackers Use

A hacker is someone who seeks and exploits the weaknesses of a computer system or network computing. Hackers may be motivated by a multitude of reasons, such as profit, protest, challenge, enjoyment or to assess these weaknesses to help in removing them.
The listed operating systems are based on the Linux kernel so it is all free operating systems.

1. Kali Linux

Kali Linux maintained and funded by Offensive Security Ltd. and it is first on our list. Kali Linux is a Debian-derived Linux distribution designed for digital forensics and penetration testing. It was developed by Mati Aharoni and Devon Kearns of Offensive Security through rewriting BackTrack, its previous forensics Linux distribution based on Ubuntu. Kali Linux has a specific project for the withdrawal of compatibility and portability of Android-specific devices, called Kali Linux NetHunter. It is the first open test platform penetration Source for Nexus Android devices, created as a joint effort between the member of the Kali "BinkyBear" Security and offensive community. It supports Wireless 802.11 frame injection, one-click configurations MANA Evil access point, keyboard HID (Teensy as attacks), as well as attacks MITM USB Mala.

2. Back Box

Back Box is an evaluation penetration testing Linux distribution and Ubuntu-based security aimed at providing an analysis of computer network systems and toolkit. Desktop environment back box includes a complete set of tools needed for ethical hacking and security testing.

3. Parrot Security OS

Parrot Security OS is a GNU / Linux distribution based on Debian. Fue built in order to perform penetration tests (safety information), vulnerability assessment and mitigation, Computer Forensics and Anonymous Surfing. Ha been developed by the team of Frozen box.
Parrot is based on the stable branch (Jessie) of Debian, a Linux 4.1 kernel hardened customized with a branch grsecurity patched available. The desktop environment is MATE fork of Gnome 2, and the default display manager is LightDM. The project is certified to run on machines with 265MB of RAM minimum follow and it is suitable for both 32-bit (i386) and 64-bit (amd64), with a special edition that works on 32-bit machines of age (486). Moreover, the project is available for Armel and armhf architectures. It even offers an edition (both 32 bit and 64 bit) developed for servers only for pen testing cloud.

4. Live Hacking OS

Live Hacking OS is a Linux distribution packed with tools and utilities for ethical hacking, penetration testing, and countermeasure verification. It includes embedded GUI GNOME user. There is a second variation available which has only the command line and requires much fewer hardware requirements.

5. DEFT Linux

DEFT stands for Digital Evidence and Forensic Toolkit and it is a distribution of Linux open source software built around the DART (Toolkit for Advanced Response Digital) and is based on the Ubuntu operating system. It has been designed from scratch to offer some of the best computer forensics open source and incident response tools that can be used by individuals, IT auditors, investigators, military, and police.

6. Samurai Web Testing Framework

The Samurai Web Testing Framework is a live Linux environment which has been pre-configured to function as a web pen-testing environment. The CD contains the best of open source and free tools that focus on testing and websites that attack. In the development of this environment, it is based on our selection of tools that we use in our practice of security. Hence, it includes the tools that were used in the four steps of a pen-test web.

7. Network Security Toolkit

The Network Security Toolkit (NST) is a Live CD based on Linux that provides a set of security tools computing and open source network to carry out routine security tasks and diagnostic networks and tracing. The distribution can be used as an analysis of network security, validation and monitoring tool for servers hosting virtual machines. NST has management capabilities similar to Fedora packages and maintains its own repository of additional packages.

8. Bugtraq

Bugtraq is a mailing list dedicated to safety issues in computers. On-topic issues new discussions about vulnerabilities, security-related notices providers, operating methods, and how to fix them. This is a mailing list of large volume, and almost all new vulnerabilities are there. Bugtraq computer freaks and experienced developers are discussed, is available in Debian, Ubuntu and openSUSE 32 and 64-bit architectures.

9. NodeZero

NodeZero is an open source system based on the operating core derived from the most popular Linux distribution in the world, Ubuntu, and designed to be used for penetration testing operations. The distribution can be downloaded as an ISO image live DVD, which will also take place on computers that support both 32-bit (x86) and 64-bit (x86_64) instruction set. Besides the fact that it allows you to start the live system, start menu contains several advanced features such as the ability to perform a diagnostic test of system memory, boot from local disk options, start the installer directly and to start in safe graphics mode, text mode or in debug mode.
Default graphical desktop environment NodeZero is powered by GNOME, which uses the classic GNOME interface. It has a design of two panels and uses the default software repositories of Ubuntu.

10. Pentoo

Pentoo is a Live CD and Live USB OS designed for penetration testing and security assessment. It is based on Gentoo Linux, Pentoo is offered both as 32-bit and 64-bit live cd which is installable. Pentoo is also available as a superposition of an existing Gentoo installation. It has conductors packet injection patched wifi, GPGPU cracking software, and plenty of tools for penetration testing and security assessment. The kernel includes Pentoo grsecurity and PAX hardening and additional patches with the binary compiled from a string of hardened with the latest nightly versions of some tools that are available.

#11 Live Hacking OS

Well, this Linux distro actually comes with some useful hacking tools which are often used in penetration testing or ethical hacking purpose. Live Hacking OS consists of the GNOME inbuilt. The operating system is really easy to operate and it can work on less RAM.

#12 Knoppix STD

This is another best Linux distro which focuses on tools for computer security. Knoppix STD brings some advanced tools for Password cracking, Firewalls, Network Utilities, Honeypots, Wireless Networking and more. This is one of the most used operating systems for Hackers.

#13 Cyborg Hawk

Cyborg Hawk is a new operating system which is based on Ubuntu Linux. Well, lots of hackers talk about Cyborg hawk and its one of the most powerful and cutting-edge penetration testing distribution that has ever been created. The operating system houses more than 700 tools for different purposes.

#14 Blackbuntu

Well, this is another operating system which is based on Linux and it was specially developed for penetration testing. Well, the operating system is very famous amongst hackers and it offers an awesome platform to learn Information security.

#15 Weakerth4n

Well, this is another best operating system which is used by professional hackers. WeakerTh4n actually comes with lots of hacking tools and it's actually a modern operating system for WiFi Hacking. Some of the wireless tools include SQL Hacking, Password Cracking, WiFi attacks, Cisco exploitation and more.
Related news

posted by safer cooler greener at 11:42 PM 0 Comments