Unfortunately no transforms exist for Danish OSINT-sources in Maltego despite Denmark being a digitally mature country7 and having a national strategy8 for publishing data to enable transparency and new business opportunities; this leads to the first deliverable of this thesis: A set of transforms to enable search of Danish OSINT-sources
It is important that the transforms fulfill the following requirements:
• Follow Paterva’s design guidelines of Maltego to seamless work with the existing transforms so users can use readily use them:
– Useful to a wide audience.
– Test the transforms (bad quality will lead to the transforms being removed from the
“Transform Hub”).
5See Sec. 2.1.2
6See Appendix A.1 for a basic explanation of Maltego.
7Ranked 11 in the world on the Networked Readiness Index 2016: http://reports.weforum.org/global-information-technology-report-2016/infographics-and-shareables/
8Denmark has joined the G8 as mentioned in the introduction.
– Error messages should be adequately verbose and set at the correct level.
– Document the use of the transforms.
– Fill out the meta-data when adding the transforms to the “Transform Hub”.
– Use API-keys correctly.
– Transforms should work out-of-the-box (or return correct error codes).
– Use standard Maltego entities9 if possible and plan design of the in- and outputs.
This is important to the overall usability of the transforms and treated separately in the design section (Sec. 4.1.1).
– Name your transforms similarly to group them in the Maltego GUI.
– Make the transforms free or with a trial.
– Remember licensing information.
– Support your users.
– Name entities consistently and such that their relation to your transform is obvious.
• Work for OSINT-sources that are expected to add value and reduce the work-load of the intelligence gathering-phase of e.g. a test. This should also encourage the pen-tester/consultant to download and use the transforms.
• Not exist on the Maltego-platform currently.
• Be supplied through the Maltego-platform for easy integration and guaranteed standardized work flows across multiple platforms.
The requirements are largely functional-only. However, tofollow design guidelines is a mix of both functional and non-functional requirements, as they both describe how the transforms should function and how they should be programmed (e.g. error messages and licensing).
Similarly, other non-functional requirements are implied by the above: Paterva offers onlyPHP and pythonfor programming to Maltego, resource management, scalability and performance are negligible in this context and managed by Maltego, and availability and similar is implied from following the design guidelines etc.
The transforms are going to be used as an integral part of the Maltego platform. Thus they need to adhere to the guidelines [44] given by Paterva (the developer); this ensures functioning, recognizable transforms and a direction for the design of the transforms. It is a descriptor of a solid quality of the transforms and desire to download and use them for the users. It is also a necessity if the transforms at a later time are to be advertised on the “Transform Hub” in Maltego (which is an advantage as it makes the transforms easier to acquire for the interested).
9These are the most basic entities always assumed to be present. They are initially added to Maltego by installing the “Paterva CTAS” transforms on the “Transform Hub”. They are also listed here: https:
//docs.paterva.com/en/entity-guide/standard_entities/.
As it can be seen the guidelines contains sound advice on designing the transforms and offering them afterwards; the concrete advice on setting them up for commercial use is not treated in this thesis, but should be referenced when maturing the transforms for commercial use in the future.
The transforms becomeuseful to a wide audience when they perform actions that can relate to many different instances of intelligence gathering. The choice of OSINT-data sources to interface with is thus important to fulfill the requirements and necessary to know before designing and implementing.
As we know from Section 2.1, both the pen-tester and the attacker, whose mind he tries to mimic, work in an agile way choosing the next step based on current findings and experience. Choosing the best sources to make the transforms for by considering all the possible paths the gathering phase can take is impossible and will likely result in spending time on building transforms that may be valuable to one pen-tester but not the next one.
Instead OSINT-sources should be chosen based on the initial knowledge we can expect the pen-tester to have. The initial knowledge can be information given directly as part of the scope of the pen-test or something known by virtually all people, e.g. the website, the company name, its general/main contact details or the location of its headquarters. Transforms already exist for crawling website content and search engines for contact details or name, so no extra value is added by making a transform for this.
In Denmark we have a transparent top level domain registry (“DK Hostmaster”), where owner data is easy to access through the public API10. This is also the case for the national register of companies “Centralt virksomhedsregister” (CVR) which provides full transparency on company information, owners, finances as well as historic changes to these11. The same national agency managing CVR also offers a public catalog of data, currently with 198 different datasets12 of varying content (e.g. location of bike parking, public transport data, road markings) published primarily by large municipalities (Copenhagen and Aarhus); Copenhagen municipality themselves currently offers 237 datasets13. This data could provide insight in a specialized pen-test but does not fulfill the requirement of a transform to have a wide audience. Similar registries exists for e.g. the national registry on property, “Bygnings- og Boligregistret” (BBR), the public registry of rights on e.g. real estate, other housing, cars, marriage contracts and personal property,
“Tinglysningsretten” and the registry of cars “Motorregistret”.
Much of the data from BBR and Tinglysningsretten is gathered and available on OIS.dk (“Den offentlige informationsserver”) and can be accessed through API’s offered by commercial part-ners14. Similarly, data on cars from Tinglysningsretten, SKAT (the national agency of taxation) and Motorregistret is offered on a commercial basis from other actors.
10https://github.com/DK-Hostmaster/whois-rest-service-specification
11Seehttp://datahub.virk.dk/dataset/system-til-system-adgang-til-cvr-data. Use requires sign-up.
12Seehttp://datahub.virk.dk/data/search
13Seehttp://data.kk.dk/
14Seehttps://ois.dk/UI/OmOIS/OmOis.aspx
The data found in all of the above registries are relevant to include in the intelligence gathering of a pen-test. Look-up in them only requires information known from the start of the engagement, thus fulfilling the requirement of being sources that are expected to add value in any intelligence gathering on any Danish target. Such transforms are not pre-existent on the platform either.
It has also been considered if the standards and guidelines in Section 2.3 could be used to select information sources to write transforms for but no pointers were found there.
Other Danish OSINT-data sources exist, but the above-mentioned can provide the required data for our transforms to follow the requirements and can thus be considered to design for.