Kasper Løvschall | Aalborg University
Using the Scopus API in your own development projects
April 4, 2017
About me
• Civil engineer (not in IT) and self-taught developer
• Working since 1998 in a research library
• First as a development consultant, subject specialist and subject coordinator of the STM area
• Today mostly in library IT with a pretty solid understanding of the internals of our “business”
• University IT today is centralised
• …but to secure domain knowledge I run my own IT department
• Been working with and experimenting with APIs for quite a long time
Final thoughts (what I have learned)
• APIs are not just for developers
• You can program with them or have “tools” use them at a higher level
• It can be a steep learning curve – but I think it is well worth the effort
• Use your domain knowledge – you might not be an IT expert but you are the one that knows how data comes together and how it can be put into play
Final thoughts (what I have learned) … continued
…
• Build on top of your competences from project to project
• Experiment, experiment and experiment before you build anything
• APIs differ greatly
- Some are programmable copies of user interfaces (yay!)
- Some are totally not
- Some drag along with a product history
- Hello! Documentation?
What is so great about APIs?
What is so great about APIs?
• Repetitive tasks
• On-demand data access
• Always up-to-date data
• No need to store data locally or in a cloud
• Combine data from different sources
• Link APIs – e.g. use data from one to query another
• Build new APIs
• Enhance a user interface with contents from the outside without the user ever knowing it (piggybacking other services)
I’m using the Scopus API in 4 systems
• LinkResolver
- One of the (if not the) most central pieces of library IT infrastructure
- Links between references / citations / metadata to the “right” full text copy of an item (e.g. article)
- The single platform where we interact with the end user – also the user that “never use the library”
- 800.000 page views / year
• SmartCoverService
- Provides cover images for our search system and LinkResolver
• PublishMe (in development)
- Provide researchers with insights on where to publish to get personal impact and where the university will get impact (university rankings)
• JournalRankings
- API providing journal metrics to e.g. LinkResolver and PublishMe
LinkResolver / SFX
• Old user interface in desperate need of some attention
• Now enhanced with metadata from 20+ different APIs
LinkResolver
• Deep linking to e.g. Scopus
LinkResolver
• Journal metrics from Scopus &
Scimago Journal Rank
LinkResolver
• Abstract and
subject keywords
LinkResolver
• Cites in Scopus with deep linking
LinkResolver
• Simple author affiliation
LinkResolver
• Simple author affiliation
Tool for hands-on experience
• Postman
• https://www.getpostman.com/
| 15
Challenges
• Expect to inherit errors or bad data from other systems
- E.g. Scopus API at some point issued illegal JSON for some searches
Challenges
• How is data serialised in the API and how you de-serialise it?
- Wikipedia: the process of translating data structures or object state into a format that can be stored (for example, in a file or memory buffer, or
transmitted across a network connection link) and reconstructed later in the same or another computer environment.
• The data is identical but the serialisation tell us something important about the underlying structure!
VS.
Challenges
• Tracking new features – even deprecation
• E.g. Scopus CiteScore killing IPP???
- New: citeScoreYearInfoList
- But no: IPPList (SNIPList and SJRList are still present)
• Un-versioned APIs
Challenges
• Unimplemented features – maybe then suddenly implemented
Challenges
• Getting around documentation – bending your brain beyond
recognition
- Read over and over, experiment, and ask for help
• “APIs are not just for developers”
- But documentation is most certainly written for developers (and maybe even by developers)
Challenges
• Getting access to the API
• Rights (when working with closed data)
- What am I allowed to do with the data
- To whom may it be displayed
- Some elements may have different rights (e.g. abstracts)
- Can data be used and stored in other systems
Final thoughts (what I have learned)
• APIs are not just for developers
• You can program with them or have “tools” use them at a higher level
• It can be a steep learning curve – but I think it is well worth the effort
• Use your domain knowledge – you might not be an IT expert but you are the one that knows how data comes together and how it can be put into play
Final thoughts (what I have learned) … continued
…
• Build on top of your competences from project to project
• Experiment, experiment and experiment before you build anything
• APIs differ greatly
- Some are programmable copies of user interfaces (yay!)
- Some are totally not
- Some drag along with a product history
- Hello! Documentation?
www.elsevier.com/research-intelligence
