From charlesreid1

Line 24: Line 24:
* Create data sets
* Create data sets
* Data engineering: gathering data process, analysis process, modeling process
* Data engineering: gathering data process, analysis process, modeling process
Charlesreid1.com data
* Logs
* Network traffic
* Loading
* Pages
* Processes


[[2018/January/Data Engineering]]
[[2018/January/Data Engineering]]

Revision as of 14:26, 15 January 2018

Task list for January:

planning stages

notes

2018/January/Notes Repositories

data engineering

Data engineering scenarios

  • Basic workflow, dealing with large amounts of data and doing machine learning on it
  • See also: Google Cloud/Review

Network data

  • Overarching goal
  • Gather data
  • Analyze trends
  • Create data sets
  • Data engineering: gathering data process, analysis process, modeling process

Charlesreid1.com data

  • Logs
  • Network traffic
  • Loading
  • Pages
  • Processes

2018/January/Data Engineering

kali

Network testbed:

  • Analyzing network traffic
  • Creating data capture workflow
  • MITM

General thread of things:

  • MITM
  • Windows exploits
  • Payload generation
  • Remote shells
  • C&C
  • Metasploitable, etc.

Git repos:

  • Setting up network testbed and instrumentation (W?)
  • Scripts for executing different attacks, etc.
  • Dropbear scripts - new org? kali?

raspberry pi

FlightAware

Microphone? Camera?

Wireless weatherproof IP camera?

master git repos

Create master repo for each organization - each repo in org as a submodule


in progress

mon jan 15

bots

master git repos

rubiks cube

2018/January/Rubiks Cube


bots

  • Tripos Bot
    • Work out small page latex
    • Get questions into latex format
    • Script to compile latex to pdf, Imagemagick latex to jpg
    • Create Rainbow Mind Machine bot
    • Create Twitter account
    • Authenticate
    • Set up and run on Rojo
    • Debug - it's just not tweeting.
  • Apollo Bot
    • Fix lengths of dialogue per speaker
    • Make lengths of dialogue per speaker a parameter/variable for function/library in rainbow mind machine
    • Get back online
  • Milton Bot
    • Seems okay... but still having some permission problem
    • Problem was, could not access keys, b/c not run from correct directory.
    • Need to fix run_bots_in_screen.sh
    • Get back online
  • Ginsberg Bot
    • Seems okay, but permission problem (see above)
    • Get back online

blog posts

  • Blog posts:
    • Knuth permutation generation
    • 4x4 Rubik's Cube: Part 1: Representations
    • 4x4 Rubik's Cube: Part 2: Permutations
    • 4x4 Rubik's Cube: Part 3: Cycles
    • Bots: Part 1: Rainbow Mind Machine, Sheep Shepherd Keymaker
    • Bots: Part 2: Poem Bot Flock
    • Bots: Part 3: Queneau Bot Flock
    • Bots: Part 4: Custom Bot Behaviors
    • Google Data Engineering Certification blog post and notes highlights
    • Data engineering scenario rollouts

genealogy

Genealogy photos:

  • Photos cropped and organized by family
  • Send email to fam with link on Dropbox (or something)

Writing:

  • Pauline and Bruce chapters
  • Historical research planning

complete

project euler

  • Project Euler:
    • Project Euler: Problem 172
    • Clean up problems 52-67

Project Euler/172

desk reorg

Reorganize and rearrange desk

  • Move book boxes to room, move table from room to main area
  • Consolidate plastic tubs
  • Computer lab - 2 laptops + desktop
  • Switch to dropbear as the main router






Tripos Bot



Flags