Scapy/Wifi Database
From charlesreid1
Overview
This page covers the use of Scapy to monitor wifi channels and populate a database with observations. These observations are completely passive and are at the physical layer (hardware) only. No network traffic.
Scripts Required
Capturing wifi data into a database will require two scripts:
- Scapy script to process wifi packets, parse information from them, send data to database
- Database script to create/connect to database, listen for data
Database Creation Script
When it comes to storing the collected wifi data in a database, you have a couple of options: 1. Store the wifi data in memory (i.e., in a Python list); the database disappears at the end of the program. This is how airomon-ng works by default. 2. Store the wifi data in a file (i.e., in a CSV file); the database is now dumped to a file and can be parsed by other programs. Like airomon-ng with the output format option set. 3. Store the wifi data in an SQL database 4. Store the wifi data in a NoSQL database
Now, let's walk through the options.
Option 1 is out - our end goal with this project will require that we capture data.
Option 2 is out - we have tried this already, and it is the last-resort option.
Option 3 is out - we have tried this already, too; works great in theory, but between the extra commas, SQL types, table schemas, and file-splitting logic, this became a total unwieldy mess.
Option 4 is in. No types to bother with - straight from Python dict to MongoDB.
Thus, the database creation script will essentially consist of setting up a MongoDB database/tables, running a test, viewing data, removing data, etc.
Wifi Capture Script
Here's how this script is going to break down:
- Functions to deal with getting wifi card information, turning it off and on, and putting it in monitor mode
- Functions to define channel hopping behavior
- Functions to filter and print out information
- Function that handles each packet
- Functions to deal with new access points and new clients
- Main function
These functions will all be assembled into a script that we'll be able to run, and have Scapy monitor the packets coming in through a wifi interface and assemble a database of wifi observations directly from those packets.
Source Code
For reference, here's the entire code we're about to examine:
import logging
logging.getLogger("scapy.runtime").setLevel(logging.ERROR) # Shut up Scapy
from scapy.all import *
conf.verb = 0 # Scapy I thought I told you to shut up
import os
import sys
import time
from threading import Thread, Lock
from subprocess import Popen, PIPE
from signal import SIGINT, signal
import argparse
import socket
import struct
import fcntl
########################################
# Begin interface settings
########################################
def get_mon_iface(args):
global monitor_on
monitors, interfaces = iwconfig()
if args.interface:
monitor_on = True
return args.interface
if len(monitors) > 0:
monitor_on = True
return monitors[0]
else:
# Start monitor mode on a wireless interface
print '['+G+'*'+W+'] Finding the most powerful interface...'
interface = get_iface(interfaces)
monmode = start_mon_mode(interface)
return monmode
def iwconfig():
monitors = []
interfaces = {}
try:
proc = Popen(['iwconfig'], stdout=PIPE, stderr=DN)
except OSError:
sys.exit('['+R+'-'+W+'] Could not execute "iwconfig"')
for line in proc.communicate()[0].split('\n'):
if len(line) == 0: continue # Isn't an empty string
if line[0] != ' ': # Doesn't start with space
wired_search = re.search('eth[0-9]|em[0-9]|p[1-9]p[1-9]', line)
if not wired_search: # Isn't wired
iface = line[:line.find(' ')] # is the interface
if 'Mode:Monitor' in line:
monitors.append(iface)
elif 'IEEE 802.11' in line:
if "ESSID:\"" in line:
interfaces[iface] = 1
else:
interfaces[iface] = 0
return monitors, interfaces
def get_iface(interfaces):
scanned_aps = []
if len(interfaces) < 1:
sys.exit('['+R+'-'+W+'] No wireless interfaces found, bring one up and try again')
if len(interfaces) == 1:
for interface in interfaces:
return interface
# Find most powerful interface
for iface in interfaces:
count = 0
proc = Popen(['iwlist', iface, 'scan'], stdout=PIPE, stderr=DN)
for line in proc.communicate()[0].split('\n'):
if ' - Address:' in line: # first line in iwlist scan for a new AP
count += 1
scanned_aps.append((count, iface))
print '['+G+'+'+W+'] Networks discovered by '+G+iface+W+': '+T+str(count)+W
try:
interface = max(scanned_aps)[1]
return interface
except Exception as e:
for iface in interfaces:
interface = iface
print '['+R+'-'+W+'] Minor error:',e
print ' Starting monitor mode on '+G+interface+W
return interface
def start_mon_mode(interface):
print '['+G+'+'+W+'] Starting monitor mode off '+G+interface+W
try:
os.system('ifconfig %s down' % interface)
os.system('iwconfig %s mode monitor' % interface)
os.system('ifconfig %s up' % interface)
return interface
except Exception:
sys.exit('['+R+'-'+W+'] Could not start monitor mode')
def remove_mon_iface(mon_iface):
os.system('ifconfig %s down' % mon_iface)
os.system('iwconfig %s mode managed' % mon_iface)
os.system('ifconfig %s up' % mon_iface)
def mon_mac(mon_iface):
'''
http://stackoverflow.com/questions/159137/getting-mac-address
'''
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
info = fcntl.ioctl(s.fileno(), 0x8927, struct.pack('256s', mon_iface[:15]))
mac = ''.join(['%02x:' % ord(char) for char in info[18:24]])[:-1]
print '['+G+'*'+W+'] Monitor mode: '+G+mon_iface+W+' - '+O+mac+W
return mac
########################################
# End interface settings
########################################
########################################
# Set channel hop behavior
########################################
def channel_hop(mon_iface, args):
'''
First time through, scan each channel for 5 seconds.
Then skip through all channels quickly.
'''
global monchannel, first_pass
channelNum = 0
maxChan = 11 if not args.world else 13
err = None
while 1:
if args.channel:
with lock:
monchannel = args.channel
else:
channelNum +=1
if channelNum > maxChan:
channelNum = 1
with lock:
first_pass = 0
with lock:
monchannel = str(channelNum)
try:
proc = Popen(['iw', 'dev', mon_iface, 'set', 'channel', monchannel], stdout=DN, stderr=PIPE)
except OSError:
print '['+R+'-'+W+'] Could not execute "iw"'
os.kill(os.getpid(),SIGINT)
sys.exit(1)
for line in proc.communicate()[1].split('\n'):
if len(line) > 2: # iw dev shouldnt display output unless there's an error
err = '['+R+'-'+W+'] Channel hopping failed: '+R+line+W
output(err, monchannel)
if args.channel:
time.sleep(.05)
else:
# For the first channel hop thru, do not deauth
if first_pass == 1:
time.sleep(1)
continue
print "this is where deauth would go:" ,monchannel
########################################
# End channel hop behavior
########################################
########################################
# Set output filtering
########################################
def output(err, monchannel):
#
# cmr:
# print out information/records as they are added to the db
#
os.system('clear')
if err:
print err
else:
print '['+G+'+'+W+'] '+mon_iface+' channel: '+G+monchannel+W+'\n'
if len(clients_APs) > 0:
print ' Clients ch ESSID'
# Print the clients list
with lock:
for ca in clients_APs:
if len(ca) > 3:
print '['+T+'*'+W+'] '+O+ca[0]+W+' - '+O+ca[1]+W+' - '+ca[2].ljust(2)+' - '+T+ca[3]+W
else:
print '['+T+'*'+W+'] '+O+ca[0]+W+' - '+O+ca[1]+W+' - '+ca[2]
if len(APs) > 0:
print '\n Access Points ch ESSID'
with lock:
for ap in APs:
print '['+T+'*'+W+'] '+O+ap[0]+W+' - '+ap[1].ljust(2)+' - '+T+ap[2]+W
print ''
def noise_filter(addr1, addr2):
ignore = ['ff:ff:ff:ff:ff:ff', # broadcast
'00:00:00:00:00:00', # broadcast
'33:33:00:', # ipv6 multicast
'33:33:ff:', # spanning tree
'01:80:c2:00:00:00', # multicast
'01:00:5e:', # broadcast
mon_MAC]
for i in ignore:
if i in addr1 or i in addr2:
return True
########################################
# End output filtering
########################################
########################################
# Set packet handling
########################################
def cb(pkt):
'''
Look for dot11 packets that aren't to or from broadcast address,
are type 1 or 2 (control, data), and append the addr1 and addr2
to the list of clients
'''
global clients_APs, APs
# We're adding the AP and channel to the clients list at time of creation rather
# than updating on the fly in order to avoid costly for loops that require a lock
if pkt.haslayer(Dot11):
if pkt.addr1 and pkt.addr2:
pkt.addr1 = pkt.addr1.lower()
pkt.addr2 = pkt.addr2.lower()
# Filter out all other APs and clients if asked
if args.accesspoint:
if args.accesspoint not in [pkt.addr1, pkt.addr2]:
return
# Check if it's added to our AP list
if pkt.haslayer(Dot11Beacon) or pkt.haslayer(Dot11ProbeResp):
APs_add(clients_APs, APs, pkt, args.channel, args.world)
# Ignore all the noisy packets like spanning tree
if noise_filter(pkt.addr1, pkt.addr2):
return
# Management = 1, data = 2
if pkt.type in [1, 2]:
clients_APs_add(clients_APs, pkt.addr1, pkt.addr2)
def APs_add(clients_APs, APs, pkt, chan_arg, world_arg):
ssid = pkt[Dot11Elt].info
bssid = pkt[Dot11].addr3.lower()
try:
# Thanks to airoscapy for below
ap_channel = str(ord(pkt[Dot11Elt:3].info))
chans = ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11'] if not args.world else ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13']
if ap_channel not in chans:
return
if chan_arg:
if ap_channel != chan_arg:
return
except Exception as e:
return
# ----------------------------------------------
# This is the money shot.
# We have an observation of a bssid, ap_channel, and ssid,
# plus we have the raw packet so we can get
# a timestamp too.
# Next step is modify this code and add it to a DB.
if len(APs) == 0:
with lock:
return APs.append([bssid, ap_channel, ssid])
else:
for b in APs:
if bssid in b[0]:
return
with lock:
return APs.append([bssid, ap_channel, ssid])
# ----------------------------------------------
def clients_APs_add(clients_APs, addr1, addr2):
# ----------------------------------------------
# This is the money shot.
# We have an observation of a client, addr1, addr2, channel, mac
# plus we have the raw packet so we can get
# a timestamp too.
# Next step is modify this code and add it to a DB.
if len(clients_APs) == 0:
with lock:
return clients_APs.append([addr1, addr2, monchannel])
# Append new clients/APs if they're not in the list
else:
for ca in clients_APs:
if addr1 in ca and addr2 in ca:
return
with lock:
return clients_APs.append([addr1, addr2, monchannel])
# ----------------------------------------------
########################################
# End packet handling
########################################
def stop(signal, frame):
if monitor_on:
sys.exit('\n['+R+'!'+W+'] Closing')
else:
remove_mon_iface(mon_iface)
os.system('service network-manager restart')
sys.exit('\n['+R+'!'+W+'] Closing')
if __name__ == "__main__":
if os.geteuid():
sys.exit('['+R+'-'+W+'] Please run as root')
clients_APs = []
APs = []
DN = open(os.devnull, 'w')
lock = Lock()
args = parse_args()
monitor_on = None
mon_iface = get_mon_iface(args)
conf.iface = mon_iface
mon_MAC = mon_mac(mon_iface)
first_pass = 1
# Start channel hopping
hop = Thread(target=channel_hop, args=(mon_iface, args))
hop.daemon = True
hop.start()
signal(SIGINT, stop)
print "sniffing"
sniff(iface=mon_iface, store=0, prn=cb)
|
Breakdown of Script
Importing Libraries
import logging
logging.getLogger("scapy.runtime").setLevel(logging.ERROR) # Shut up Scapy
from scapy.all import *
conf.verb = 0 # Scapy I thought I told you to shut up
import os
import sys
import time
from threading import Thread, Lock
from subprocess import Popen, PIPE
from signal import SIGINT, signal
import argparse
import socket
import struct
import fcntl
Handling Wifi Interfaces
Here, we have a collection of functions that deal with the wifi interfaces:
########################################
# Begin interface settings
########################################
def get_mon_iface(args):
global monitor_on
monitors, interfaces = iwconfig()
if args.interface:
monitor_on = True
return args.interface
if len(monitors) > 0:
monitor_on = True
return monitors[0]
else:
# Start monitor mode on a wireless interface
print '['+G+'*'+W+'] Finding the most powerful interface...'
interface = get_iface(interfaces)
monmode = start_mon_mode(interface)
return monmode
def iwconfig():
monitors = []
interfaces = {}
try:
proc = Popen(['iwconfig'], stdout=PIPE, stderr=DN)
except OSError:
sys.exit('['+R+'-'+W+'] Could not execute "iwconfig"')
for line in proc.communicate()[0].split('\n'):
if len(line) == 0: continue # Isn't an empty string
if line[0] != ' ': # Doesn't start with space
wired_search = re.search('eth[0-9]|em[0-9]|p[1-9]p[1-9]', line)
if not wired_search: # Isn't wired
iface = line[:line.find(' ')] # is the interface
if 'Mode:Monitor' in line:
monitors.append(iface)
elif 'IEEE 802.11' in line:
if "ESSID:\"" in line:
interfaces[iface] = 1
else:
interfaces[iface] = 0
return monitors, interfaces
def get_iface(interfaces):
scanned_aps = []
if len(interfaces) < 1:
sys.exit('['+R+'-'+W+'] No wireless interfaces found, bring one up and try again')
if len(interfaces) == 1:
for interface in interfaces:
return interface
# Find most powerful interface
for iface in interfaces:
count = 0
proc = Popen(['iwlist', iface, 'scan'], stdout=PIPE, stderr=DN)
for line in proc.communicate()[0].split('\n'):
if ' - Address:' in line: # first line in iwlist scan for a new AP
count += 1
scanned_aps.append((count, iface))
print '['+G+'+'+W+'] Networks discovered by '+G+iface+W+': '+T+str(count)+W
try:
interface = max(scanned_aps)[1]
return interface
except Exception as e:
for iface in interfaces:
interface = iface
print '['+R+'-'+W+'] Minor error:',e
print ' Starting monitor mode on '+G+interface+W
return interface
def start_mon_mode(interface):
print '['+G+'+'+W+'] Starting monitor mode off '+G+interface+W
try:
os.system('ifconfig %s down' % interface)
os.system('iwconfig %s mode monitor' % interface)
os.system('ifconfig %s up' % interface)
return interface
except Exception:
sys.exit('['+R+'-'+W+'] Could not start monitor mode')
def remove_mon_iface(mon_iface):
os.system('ifconfig %s down' % mon_iface)
os.system('iwconfig %s mode managed' % mon_iface)
os.system('ifconfig %s up' % mon_iface)
def mon_mac(mon_iface):
'''
http://stackoverflow.com/questions/159137/getting-mac-address
'''
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
info = fcntl.ioctl(s.fileno(), 0x8927, struct.pack('256s', mon_iface[:15]))
mac = ''.join(['%02x:' % ord(char) for char in info[18:24]])[:-1]
print '['+G+'*'+W+'] Monitor mode: '+G+mon_iface+W+' - '+O+mac+W
return mac
########################################
# End interface settings
########################################
Channel Hopping Behavior
We define a function to define how Scapy will hop through channels:
########################################
# Set channel hop behavior
########################################
def channel_hop(mon_iface, args):
'''
First time through, scan each channel for 5 seconds.
Then skip through all channels quickly.
'''
global monchannel, first_pass
channelNum = 0
maxChan = 11 if not args.world else 13
err = None
while 1:
if args.channel:
with lock:
monchannel = args.channel
else:
channelNum +=1
if channelNum > maxChan:
channelNum = 1
with lock:
first_pass = 0
with lock:
monchannel = str(channelNum)
try:
proc = Popen(['iw', 'dev', mon_iface, 'set', 'channel', monchannel], stdout=DN, stderr=PIPE)
except OSError:
print '['+R+'-'+W+'] Could not execute "iw"'
os.kill(os.getpid(),SIGINT)
sys.exit(1)
for line in proc.communicate()[1].split('\n'):
if len(line) > 2: # iw dev shouldnt display output unless there's an error
err = '['+R+'-'+W+'] Channel hopping failed: '+R+line+W
output(err, monchannel)
if args.channel:
time.sleep(.05)
else:
# For the first channel hop thru, do not deauth
if first_pass == 1:
time.sleep(1)
continue
print "this is where deauth would go:" ,monchannel
########################################
# End channel hop behavior
########################################
Output Filtering
Here we define what the script will print to the screen as it proceeds, and what packets it will filter out as irrelevant (NOTE: This will change quite a bit from its current state):
########################################
# Set output filtering
########################################
def output(err, monchannel):
#
# cmr:
# print out information/records as they are added to the db
#
os.system('clear')
if err:
print err
else:
print '['+G+'+'+W+'] '+mon_iface+' channel: '+G+monchannel+W+'\n'
if len(clients_APs) > 0:
print ' Clients ch ESSID'
# Print the clients list
with lock:
for ca in clients_APs:
if len(ca) > 3:
print '['+T+'*'+W+'] '+O+ca[0]+W+' - '+O+ca[1]+W+' - '+ca[2].ljust(2)+' - '+T+ca[3]+W
else:
print '['+T+'*'+W+'] '+O+ca[0]+W+' - '+O+ca[1]+W+' - '+ca[2]
if len(APs) > 0:
print '\n Access Points ch ESSID'
with lock:
for ap in APs:
print '['+T+'*'+W+'] '+O+ap[0]+W+' - '+ap[1].ljust(2)+' - '+T+ap[2]+W
print ''
def noise_filter(addr1, addr2):
ignore = ['ff:ff:ff:ff:ff:ff', # broadcast
'00:00:00:00:00:00', # broadcast
'33:33:00:', # ipv6 multicast
'33:33:ff:', # spanning tree
'01:80:c2:00:00:00', # multicast
'01:00:5e:', # broadcast
mon_MAC]
for i in ignore:
if i in addr1 or i in addr2:
return True
########################################
# End output filtering
########################################
Packet Handling
This is the most important function! Each time the wifi interface receives a packet, it will run the following function on that packet:
########################################
# Set packet handling
########################################
def cb(pkt):
'''
Look for dot11 packets that aren't to or from broadcast address,
are type 1 or 2 (control, data), and append the addr1 and addr2
to the list of clients
'''
global clients_APs, APs
# We're adding the AP and channel to the clients list at time of creation rather
# than updating on the fly in order to avoid costly for loops that require a lock
if pkt.haslayer(Dot11):
if pkt.addr1 and pkt.addr2:
pkt.addr1 = pkt.addr1.lower()
pkt.addr2 = pkt.addr2.lower()
# Filter out all other APs and clients if asked
if args.accesspoint:
if args.accesspoint not in [pkt.addr1, pkt.addr2]:
return
# Check if it's added to our AP list
if pkt.haslayer(Dot11Beacon) or pkt.haslayer(Dot11ProbeResp):
APs_add(clients_APs, APs, pkt, args.channel, args.world)
# Ignore all the noisy packets like spanning tree
if noise_filter(pkt.addr1, pkt.addr2):
return
# Management = 1, data = 2
if pkt.type in [1, 2]:
clients_APs_add(clients_APs, pkt.addr1, pkt.addr2)
def APs_add(clients_APs, APs, pkt, chan_arg, world_arg):
ssid = pkt[Dot11Elt].info
bssid = pkt[Dot11].addr3.lower()
try:
# Thanks to airoscapy for below
ap_channel = str(ord(pkt[Dot11Elt:3].info))
chans = ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11'] if not args.world else ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13']
if ap_channel not in chans:
return
if chan_arg:
if ap_channel != chan_arg:
return
except Exception as e:
return
# ----------------------------------------------
# This is the money shot.
# We have an observation of a bssid, ap_channel, and ssid,
# plus we have the raw packet so we can get
# a timestamp too.
# Next step is modify this code and add it to a DB.
if len(APs) == 0:
with lock:
return APs.append([bssid, ap_channel, ssid])
else:
for b in APs:
if bssid in b[0]:
return
with lock:
return APs.append([bssid, ap_channel, ssid])
# ----------------------------------------------
def clients_APs_add(clients_APs, addr1, addr2):
# ----------------------------------------------
# This is the money shot.
# We have an observation of a client, addr1, addr2, channel, mac
# plus we have the raw packet so we can get
# a timestamp too.
# Next step is modify this code and add it to a DB.
if len(clients_APs) == 0:
with lock:
return clients_APs.append([addr1, addr2, monchannel])
# Append new clients/APs if they're not in the list
else:
for ca in clients_APs:
if addr1 in ca and addr2 in ca:
return
with lock:
return clients_APs.append([addr1, addr2, monchannel])
# ----------------------------------------------
########################################
# End packet handling
########################################
Stop Wifi Interfaces
Stop the wifi interfaces:
def stop(signal, frame):
if monitor_on:
sys.exit('\n['+R+'!'+W+'] Closing')
else:
remove_mon_iface(mon_iface)
os.system('service network-manager restart')
sys.exit('\n['+R+'!'+W+'] Closing')
Main Function
FINALLY, we can get to the main function!
if __name__ == "__main__":
if os.geteuid():
sys.exit('['+R+'-'+W+'] Please run as root')
clients_APs = []
APs = []
DN = open(os.devnull, 'w')
lock = Lock()
args = parse_args()
monitor_on = None
mon_iface = get_mon_iface(args)
conf.iface = mon_iface
mon_MAC = mon_mac(mon_iface)
first_pass = 1
# Start channel hopping
hop = Thread(target=channel_hop, args=(mon_iface, args))
hop.daemon = True
hop.start()
signal(SIGINT, stop)
print "sniffing"
sniff(iface=mon_iface, store=0, prn=cb)
Flags
 |
scapy a Python library for interfacing with network devices and analyzing packets from Python.
Building Wireless Utilities: Scapy/Airodump Clone · Scapy/AP Scanner Analyzing Conversations: Scapy/Conversations Database: Scapy/Wifi Database Category:Scapy · Category:Python · Category:Networking
|
|
Wireless all things wireless.
Software:
|