Big data is used for many purposes, usually to streamline business operations or increase competitive advantage. But it can also help make the world a better place. To this end, IST Research is using big data to combat illicit behaviors and improve humanitarian response efforts around the world.
Typically, data is generated by people who use technologies such as mobile apps, online ads, social media and the Internet generally. But not all parts of the world have access to smartphones or reliable Internet connectivity. IST Research specializes in finding ways to generate and collect data in challenging environments with populations that are outside the reach of these technologies, ensuring that the voice of marginalized people is reflected in the ‘big data’ that is used to inform resource allocation and policy making.
Human trafficking and big data
As one of its many initiatives, IST Research creates innovative technologies to fight human trafficking.
Search engines can only index a small percentage of sites on the surface web, while much of the deep web – such as dynamically generated or temporary sites – remains unsearchable. Even more inaccessible is the dark web, where anonymous sites with untraceable IP addresses are sometimes used for criminal activity. Specialized browsers allow access to these ‘dark web’ sites that can’t be indexed by typical commercial search engines.
IST Research collects and stores data from all parts of the web, including the surface web, deep web and the dark web, and provides data and analytics to anti-trafficking actors – including law enforcement, public prosecutors, and NGOs – to help them investigate and prosecute these hidden crimes more efficiently and effectively.
Data collection and analysis
IST Research uses multiple approaches to collect data and then aggregates it to get a broader view of activity within a geographic region. (See “Combining Actively and Passively Collected Data to Create Meaningful Results” to learn more.)
After collection, IST Research uses Internap infrastructure for data storage and processing. For efficient analysis, the company requires a highly configurable infrastructure with high IOPS and network performance for fast data transfer.
Bare metal
The company uses Internap’s bare-metal AgileSERVER based in the Secaucus, NJ data center. Bare metal was the clear choice for IST Research to achieve high performance in a cost-effective manner. The performance benefits of bare metal allow the company to provide more relevant and accurate intelligence in near real-time to aid counter-trafficking programs.
Prior to Internap, IST Research used Amazon Web Services (AWS) for their infrastructure needs, but after evaluating multiple vendors including AWS and Rackspace, none could match the same level of performance as Internap.
High IOPS
IST Research requires rapid data transfer throughout its infrastructure to conduct efficient analysis. Internap’s bare-metal AgileSERVER and Managed Internet Route OptimizerTM (MIRO) technology allows IST Research to meet its IOPS requirements and ultimately reduce data transfer time by more than 30%.
Configuration management tools & support
Unlike larger commodity cloud solutions, Internap allows IST Research to change configurations within hours. Dedicated account and support teams work in conjunction with the IST Research team to ensure optimal infrastructure configurations at all times.
“The Edge”
The Edge is the term IST Research uses to refer to places without reliable Internet connectivity, power or other support resources. Those who live and work at the edge have traditionally been outside the reach of big data, but sometimes these are the places that need the most help. By creating new ways to engage with populations that live at the Edge, IST Research may indeed change the world through technology.
