Frequently Asked Questions

Last updated: 01.08.2024

Why Search Smart?

Why should I use Search Smart?

With Search Smart, you find with confidence via the most productive databases.

Search Smart suggests the best databases for your purpose based on a comprehensive comparison of most of the popular English academic databases. Search Smart tests the critical functionalities databases offer. Thereby, we uncover the capabilities and limitations of search systems that are not reported anywhere else. Search Smart aims to provide the best – i.e., most accurate, up-to-date, and comprehensive – information possible on search systems’ functionalities.

Researchers use Search Smart as a decision tool to select the system/database that fits best.

Librarians use Search Smart for giving search advice and for procurement decisions.

Search providers use Search Smart for benchmarking and improvement of their offerings.

How is Search Smart innovative?

Four novel ideas and methods enable Search Smart:

Adaptation of the 'Query Hit Count' method using different types of queries allowing the assessment of different academic database types. A big shout-out goes to the members of the EC3 Research Group at the Universidad de Granada for the groundwork and inspiration! Research ideas that helped build Search Smart includes some groundwork on query hit data by Orduña-Malea et al. (2014) and database comparisons by José Luis Ortega (2014).
Application of 'metamorphic testing' to academic databases allowing their assessment despite limited data availability. Over the years, we developed and tested various metamorphic relations that help assess the qualities and limitations of specific search functionalities. At a basic level, these relations test how variations in the search queries translate into changes in the search results. To learn about some basic principles of 'metamorphic testing' you can read, e.g., Segura et al. (2020). For more information on the tests we perform, read the FAQ section 'Our testing'.
Introduction of the 'Search Triangle’ notion to academic search, specifying that databases and heuristics need to be matched to the goals different search types have.
Introduction of the 'Basket of Keywords' method allowing the assessment of subject coverage of a large number of databases.

The associated (open access) publications to each of these ideas and methods are the following:

Gusenbauer, M. (2019). Google Scholar to Overshadow Them All? Comparing the Sizes of 12 Academic Search Engines and Bibliographic Databases. Scientometrics, 118(1), 177–214. https://doi.org/10.1007/s11192-018-2958-5 [direct download]
Gusenbauer, M., & Haddaway, N. R. (2020). Which Academic Search Systems are Suitable for Systematic Reviews or Meta-Analyses? Evaluating Retrieval Qualities of Google Scholar, PubMed and 26 other Resources. Research Synthesis Methods, 11(2), 181–217. https://doi.org/10.1002/jrsm.1378 [direct download]
Gusenbauer, M., & Haddaway, N. R. (2021). What every Researcher should know about Searching – Clarified Concepts, Search Advice, and an Agenda to improve Finding in Academia. Research Synthesis Methods, 12(2), 136–147. https://doi.org/10.1002/jrsm.1457 [direct download]
Gusenbauer, M. (2022). Search where you will find most: Comparing the disciplinary coverage of 56 bibliographic databases. Scientometrics, 127, 2683–2745. https://doi.org/10.1007/s11192-022-04289-7 [direct download]

References:

Orduña-Malea, E., Ayllón, J. M., Martín-Martín, A., & Delgado López-Cózar, E. (2014). About the size of Google Scholar: playing the numbers. EC3 Working Papers, 18(23).
Ortega, J. L. (2014). Academic Search Engines: A quantitative outlook. Chandos information professional series. Chandos Publishing/Elsevier.
Segura, S., Towey, D., Zhou, Z. Q., & Chen, T. Y. (2020). Metamorphic Testing: Testing the Untestable. IEEE Software, 37(3), 46–53. https://doi.org/10.1109/MS.2018.2875968

What databases do you compare at the moment?

At the moment (as of August 1st, 2024), we cover 106 databases and their search systems. Here is the list, ranked alphabetically:

ABI/Inform Global (via ProQuest)
Academia.edu
Academic Search Elite (via EBSCOhost)
Academic Search Premier (via EBSCOhost)
ACM Guide to Computing Literature
ACS Publications
AIS eLibrary
APA PsycInfo (via EBSCOhost)
APA PsycInfo (via Ovid)
APA PsycNet
Arts & Humanities Citation Index (via Web of Science)
arXiv
Bielefeld Academic Search Engine
Biological Science Database (via ProQuest)
BIOSIS Citation Index (via Web of Science)
Business Source Premier (via EBSCOhost)
CAB Abstracts (via Ovid)
CAS SciFinder-n
CINAHL (via EBSCOhost)
CINAHL Plus (via EBSCOhost)
ClinicalTrials.gov
CNKI Overseas
Cochrane Database of Systematic Reviews
Cochrane Library - CENTRAL
Conference Proceedings Citation Index - Science (via Web of Science)
Conference Proceedings Citation Index - Social Science & Humanities (via Web of Science)
CORE
Crossref
dblp
Dimensions - Clinical Trials
Dimensions - Publications
Dimensions - Publications (free)
Dissertations & Theses Global (via ProQuest)
DOAJ
Earth, Atmospheric & Aquatic Science Database (via ProQuest)
EconBiz
EconLit (via EBSCOhost)
EconStor
Embase (via Ovid)
Emerald Insight
Emerging Sources Citation Index (via Web of Science)
Environmental Science Database (via ProQuest)
Epistemonikos
ERIC
ERIC (via EBSCOhost)
EU Clinical Trials Register
Europe PMC
Food Science and Technology Abstracts (via EBSCOhost)
GeoRef (via ProQuest)
Google Scholar
GreenFILE (via EBSCOhost)
HAL
IEEE Xplore
Ingenta Connect
International Bibliography of the Social Sciences (via ProQuest)
Internet Archive Scholar
JSTOR
Lens
Medline (via EBSCOhost)
Medline (via Ovid)
Medline (via Web of Science)
Mendeley
Naver Academic
Nursing & Allied Health Database (via ProQuest)
Open Access Theses and Dissertations
OpenAIRE
OpenAlex
Overton - Scholarly Articles
Overton - Policy Documents
Paperity
Pascal and Francis Bibliographic Databases
Policy Commons
Psychology & Behavioral Sciences Collection (via EBSCOhost)
Public Health Database (via ProQuest)
Public Library of Science (PLOS)
PubMed
RePEc (via EconPapers)
RePEc (via IDEAS)
ResearchGate
SAGE Journals Online
ScanMedicine
Science Citation Index Expanded (via Web of Science)
ScienceDirect
ScienceOpen
Scilit
Scinapse
Scite
SciTech Premium Collection (via ProQuest)
Scopus
Semantic Scholar
Social Sciences Citation Index (via Web of Science)
Social Sciences Premium Collection (via ProQuest)
SocINDEX (via EBSCOhost)
Sociological Abstracts (via ProQuest)
SPORTDiscus (via EBSCOhost)
SpringerLink
SSRN
Taylor and Francis Online
Virtual Health Library
Web of Science Core Collection
WHO International Clinical Trials Registry Platform
Wiley Online Library
World Transit Research
WorldCat - Article/chapter only
WorldCat - Thesis/dissertation
zbMATH Open

Databases we explicitly decided NOT to cover (see selection criteria under 'our testing'):

ArnetMiner (QHC data issues)
OSF Preprints (QHC data issues)
Scielo (non-English focus)
Transport Research International Documentation (QHC data issues)
WorldWideScience (QHC data issues)

Why is your testing procedure sound and valid?

Our testing methods have undergone peer review, and all of them are transparently explained in the FAQ 'our testing'. We have developed these logics over several years and published the exact steps, advantages, and limitations in academic journals. These publications have received significant attention. Search Smart is the user-friendly amalgamation of experiences we have collected from testing all different kinds of academic databases over the years.

Search Smart data means a significant step towards better transparency in academic search. Nevertheless, the data has some limitations: Coverage data are not always exact, and sometimes they are (good) estimates of what a database contains. Better assessments would be possible with full access to databases, which in most cases is impossible due to database providers' data policies. The functionalities tests show when a search system (that makes databases accessible) fails or not. We cannot rule out that the system would fail in other tests. Thus, we cannot guarantee with certainty that a system always performs as expected. To validate such functionality, one would need access to the system, which again is impossible. Using these testing methods, Search Smart can complement and independently validate search help and feature information offered by search providers. Too often, this information is incorrect or incomplete.

This is the list of (open access) publications Search Smart's methodology primarily builds upon:

Gusenbauer, M. (2019). Google Scholar to Overshadow Them All? Comparing the Sizes of 12 Academic Search Engines and Bibliographic Databases. Scientometrics, 118(1), 177–214. https://doi.org/10.1007/s11192-018-2958-5 [direct download]
Gusenbauer, M., & Haddaway, N. R. (2020). Which Academic Search Systems are Suitable for Systematic Reviews or Meta-Analyses? Evaluating Retrieval Qualities of Google Scholar, PubMed and 26 other Resources. Research Synthesis Methods, 11(2), 181–217. https://doi.org/10.1002/jrsm.1378 [direct download]
Gusenbauer, M., & Haddaway, N. R. (2021). What every Researcher should know about Searching – Clarified Concepts, Search Advice, and an Agenda to improve Finding in Academia. Research Synthesis Methods, 12(2), 136–147. https://doi.org/10.1002/jrsm.1457 [direct download]
Gusenbauer, M. (2022). Search where you will find most: Comparing the disciplinary coverage of 56 bibliographic databases. Scientometrics, 127, 2683–2745. https://doi.org/10.1007/s11192-022-04289-7 [direct download]

What does Search Smart cost me?

Search Smart is entirely free to use.

Specific advice

Should I search one or multiple databases?

Search Smart lists the databases that are most comprehensive in specific subjects. Depending on your search goals (lookup, exploratory, or systematic searching), it will make more sense to search one or multiple databases. In quick lookup searches, one comprehensive database containing your answer is enough. When your goal is to explore or identify all records on a specific topic searching multiple databases is advisable. In exploration, you may go for larger, multidisciplinary databases that allow good navigation and citation searching. New ways to sort search results will help you identify interesting results. E.g., you may sort via citation type, or the attention-specific records received, in addition to pure relevance-based rankings.

Particularly in systematic searching, where the goal is to identify all records on a topic in a transparent and reproducible manner, searching multiple databases is critical. Many search guides advise using at least two or better three or more databases to get a good search yield. In selecting databases, you might want to use a combination of large, multidisciplinary ones with some specialization in your topic of interest. It is essential to include databases that offer diversity in record types (including gray literature) and search heuristics. Effective searches will consist of both Boolean and (backward/forward) citation searching. With Search Smart, you can filter for the best options for you among the most important and popular databases in academia. Yet, for your specific question, there might still be additional databases beyond the scope available on Search Smart.

How do I get a sorted list of the best databases?

To find the best database for you...

MUST: Select the subject(s) that are relevant to you.
MUST: Filter system functionality that is important for you either via one of our three presets or via the many options provided in the filter list.
MUST: Sort the best options according to your needs: 'most coverage' ('total coverage'; 'abs. subj. cov.'; 'record type coverage') vs. 'most specialized' ('rel. subj. cov.') in your subject, etc.
OPTIONAL: If you do not have institutional access to paywalled systems (e.g., ProQuest, EBSCOhost, Ovid, Web of Science), select 'Non-paywalled databases', and freely accessible options will show.

Pro Tip 1: pin two to six databases and compare them in detail.

Pro Tip 2: download a PDF-save of your selection for future reference and have a permanent link to re-visit your current preferences. This way, you can store what you found and update it later to incorporate changes that occurred in the meantime.

Pro Tip 3: use one of the three presets to quickly get an overview of which systems best fit your chosen search type.

For step-by-step instructions on specific use cases, please navigate to Search Smart's tutorials section.

How should I report my search process?

Adequate reporting of how databases were searched is essential for transparency and reproducibility. Particularly in research designs that are geared towards systematically synthesizing information, it is important to know where this information came from and what the situation was when the database was accessed. Not only systematic literature reviews require accurate reporting. It should be important for any researcher to keep track of how information was accessed throughout the research process. To adequately report the databases you access, you need to state:

What: the database(s) you accessed and the search system(s) that provided access to the database(s). Sometimes the database is the same as the search system (e.g., 'SCOPUS'), and sometimes the database (e.g., Medline) can be accessed through different search systems (e.g., 'EBSCOhost', 'Web of Science'). is (this is sometimes the same as the database itself: e.g., 'SCOPUS' is the name of the database and the system, yet 'Embase' is the name of the database. ATTENTION: particularly 'Web of Science' delivers different versions to institutions that mostly differ in their retrospective coverages. Thus, always report the underlying indices you searched. The 'Web of Science Core Collection' consists of a varying number and quality of databases that is different for each university.
When: date of search
How: the exact search query that you used (incl. keywords, operators, field codes) and the filters you employed
How many: how many records did you identify (keep a copy of the export of records for future reference)

The 2020 PRISMA statement (https://prisma-statement.org/) helps you to adequately report the search process in systematic reviews and meta-analyses.

How do I know which databases I filtered out?

You can export a PDF of databases you selected with your filters and criteria. This PDF will contain a list of database and the details of the criteria it did not meet.

How to download a report of my Search Smart results?

After you have received a sorted list of suitable databases for you, you may download it as a PDF for future reference. In this PDF, you see all databases that met your criteria and which ones did not and why. Your selection is also saved there so that you can update your selection at any time in the future. As Search Smart updates information periodically, you may find different databases that meet your criteria in the future.

Follow these steps at Search Smart:

After you selected all the criteria important to you...
Click 'Save selection to PDF'
Done. Review the sorted results with all search filters you selected.

Presets

What are presets?

Search Smart offers three pre-defined filter settings, so-called presets, to identify the best database(s) for you:

'Systematic keyword searching'
'Backward citation searching'
'Forward citation searching'

Presets are a quick way to filter databases for specific search requirements. They give a good first selection of databases and are a good way to start thinking about the critical capabilities systems need to offer. These presets filter databases based on each search type's specific requirements. Filtering with presets will activate multiple filters that jointly form the minimum capabilities a database and its search system must fulfill to be deemed 'sufficiently' suitable. Users may additionally select the 'bulk select & export options' button to activate additional filters that limit the selection to options with extensive retrieval qualities. These databases' capabilities are rated 'extensive'. If databases fail to fulfill a minimum requirement, they are rated 'limited' and cannot be used for this type of searching or should only be used with great caution (depending on the specific limitation). Details for the specific search types can be found in the FAQs: presets.

Users may have additional requirements to the ones selected with presets. Then, they simply add additional filters to adjust the selection of databases to their requirements. Experimenting with activating/deactivating filter and preset settings is straightforward, allowing users to learn which databases support which functionalities.

PRESET 1: When to select 'systematic keyword searching'?

Systematic keyword searching is the most critical element of search strategies in most systematic reviews and meta-analyses. Its goal is to identify all relevant records that meet the eligibility criteria in a reproducible and transparent manner. Boolean searching, mostly relying on OR/AND operators, is the foundation of any systematic search. It yields many relevant results when performed well with the right database and the right search string. Moreover, Boolean searching proves highly reproducible and transparent, but only with the suitable search systems and good reporting. However, only a fraction of databases support key functionalities for systematic searching. Search Smart allows to easily identify the databases that support critical functionality. To warrant the criteria for 'rigorous research' in systematic reviews, a database needs to fulfill nine minimum capabilities that are enabled all at once via the "systematic keyword searching" preset:

Minimum search string length (narrow field code) is 25 or more
Verbatim queries
Reproducible queries over time/place
Boolean OR
Boolean AND
Boolean operators work exactly
Field code "abstract"
Nested search (parenthesis)
Accessible records: 1000 or more (systematic searches will, in most cases, go well beyond the first results page)

These criteria are based on the ones developed here: Gusenbauer, M., & Haddaway, N. R. (2020). Which Academic Search Systems are Suitable for Systematic Reviews or Meta-Analyses? Evaluating Retrieval Qualities of Google Scholar, PubMed and 26 other Resources. Research Synthesis Methods, 11(2), 181–217. https://doi.org/10.1002/jrsm.1378 [direct download]

Additionally, selecting the “Bulk select & export options” button will activate two more filters that further limit the selection by requiring databases to support:

Bulk select records
Bulk export records: 50 or more at a time

Users with advanced requirements may limit the selection further with additional filtering options: e.g., truncation, Boolean NOT operator, search history, query builder, proximity operators, or support for exceptionally long queries.

PRESET 2: When to select 'backward citation searching'?

A way to search systematically is by 'chasing' all backward citations of selected seed records. This way, reviewers might identify additional relevant records that link to an already identified relevant record. Particularly for topics with diverse language that is hard to capture by keyword searches alone, citation searching may be of help. For a search database to be suitable for backward citation searching, it must provide the following minimum capabilities that are enabled at once via the "backward citation searching" preset:

Backward citation information
Accessible records: 1000 or more

Additionally, selecting the “Bulk select & export options” button will activate two more filters that further limit the selection by requiring databases to support:

Bulk select records' backward citations
Bulk export backward citations: 500 or more at a time

The 'backward citation searching' preset works best with the 'Backward citation score' sorting selection. The backward citation score is a quantitative metric that considers (1) the approximate share of documents with citation information and (2) the accuracy of the citation information.

PRESET 3: When to select 'forward citation searching'?

A way to search systematically is by 'chasing' all forward citations of selected seed records. This way, reviewers might identify additional relevant records that link to an already identified relevant record. Particularly for topics with diverse language that is hard to capture by keyword searches alone, citation searching may be of help. For a search database to be suitable for forward citation searching, it must provide the following minimum capabilities that are enabled at once via the "forward citation searching" preset:

Forward citation information
Accessible records: 1000 or more

Additionally, selecting the “Bulk select & export options” button will activate two more filters that further limit the selection by requiring databases to support:

Bulk select records' forward citations
Bulk export forward citations: 500 or more at a time

The 'forward citation searching' preset works best with the 'Forward citation score' sorting selection. The forward citation score is a quantitative metric that considers (1) the approximate share of documents with citation information and (2) the completeness of the citation information.

Functions

Coverage filters

Search Smart filters out databases NOT COVERING a selected coverage type OR when the coverage info is UNAVAILABLE. Search Smart assumes that if you filter something, then it is important to you. Thus, we only list the databases where we know they cover what is relevant to you.

Search Smart offers you five different coverage types you may filter databases with:

Subject coverage: share of records from a specific subject
Keyword coverage: prevalence of records including specific keywords in the title
Record type coverage: share of records from a specific type
Retrospective coverage: share of records from a specific time range
Open access coverage: share of open access records

Search functions filters

Search Smart offers you many options to filter for your ideal database. These fall into ten different search and retrieval categories:

Interface: some basic search options the search interface offers
Sorting options: how you may sort search results on the database
Query: how well you can perform queries on a database
Field codes: what field codes you may use to narrow your search to specific areas/meta-data of a record
Operators: what operators you may use to construct a keyword query and whether they work as expected
Pre-/Post-query filters: what filter types (facets) the search interface offers to narrow down your search results
Record type filters: what record types you may filter the search results with
Citation search: what types of citation searches the database offers (incl. suggestions of related records)
Retrieval: what retrieval options the database offers
Export formats: what formats search results may be exported with

Sorting options

Search Smart offers sorting options that allow you to see the best-performing databases on top. For some sorting, you need first to specify the type of coverage, so Search Smart knows what is important to you (e.g., the specific subjects that should be sorted). The sorting options in bold are always available in the scholarly records view. The others only appear when subjects, keywords, or records types are selected first:

Alphabetically (A-Z)
Alphabetically (Z-A)
Total coverage (descending) - called 'clinical trials coverage' in 'clinical trials' view
Subject coverage (absolute / descending)
Subject coverage (relative / descending)
Keyword coverage (absolute / descending)
Keyword coverage (relative / descending)
Record type coverage (absolute / descending)
Record type coverage (relative / descending)
Subject x Record type coverage (absolute / descending)
Subject x Record type coverage (relative / descending)
Open access coverage (descending) - not available in 'clinical trials' view

Detailed view

Search Smart provides a detailed view of every database it analyzes. The detailed view shows:

General information about the database, like its owner, headquarters, year of launch, or a verbal description from the system providers.
Retrospective coverage illustrated by a line chart.
Subject coverage illustrated by a donut chart.
Record type coverage illustrated by a donut chart.
Presets assessment: systematic keyword searching, forward citation searching, grey literature searching.
Detailed information on search functionalities: interface, query, operators, citations search/filtering, retrieval.

Tooltips inform about important system characteristics or when the information was last updated.

Pop-ups detail the quantitative information of certain functionalities (e.g., which specific field codes are supported by the system). This information reflects the verbatim names used by the database. Thus, in many cases you can directly apply this info to your searches on this database.

Pin & compare

To compare up to six databases you can pin them and press the yellow 'compare' button. Change your selection on the go and find out which databases best fits your purpose.

Save filter settings

You can save your filter settings by copying the URL in the browser tab or using the 'share' button. This will automatically save the filters and sorting options you selected. You can re-use these filters again to account for the changes that occurred in systems and databases in the meantime. As databases update and functionalities change, your list of best databases will change too. So keep revisiting Search Smart to find out about the latest changes and new databases that might do your job even better.

You can either copy the URL directly, share it with peers or extract it from a PDF you created to save your selection of databases.

Save PDF

Saving a PDF with your selection of databases helps you to store the state of the search system and database landscape captured by Search Smart at that time. You can append the document in reviews and research papers to justify database selection. Such export will reflect the filter criteria that were important for you at that specific point in time and the state of search systems and databases were in.

Moreover, the specific filter settings of your selection are also stored in the PDF. This way you can re-evaluate your selection of databases after some time and incorporate changes that occurred in the meantime.

Cite

To acknowledge the help of Search Smart, please cite it!

The tool:

Gusenbauer, M. (2023). A free online guide to researchers’ best search options. Nature, 615, 586. https://doi.org/10.1038/d41586-023-00845-0

Search Smart is built upon the following studies:

Gusenbauer, M. (2019). Google Scholar to Overshadow Them All? Comparing the Sizes of 12 Academic Search Engines and Bibliographic Databases. Scientometrics, 118(1), 177–214. https://doi.org/10.1007/s11192-018-2958-5 [direct download]
Gusenbauer, M., & Haddaway, N. R. (2020). Which Academic Search Systems are Suitable for Systematic Reviews or Meta-Analyses? Evaluating Retrieval Qualities of Google Scholar, PubMed and 26 other Resources. Research Synthesis Methods, 11(2), 181–217. https://doi.org/10.1002/jrsm.1378 [direct download]
Gusenbauer, M., & Haddaway, N. R. (2021). What every Researcher should know about Searching – Clarified Concepts, Search Advice, and an Agenda to improve Finding in Academia. Research Synthesis Methods, 12(2), 136–147. https://doi.org/10.1002/jrsm.1457 [direct download]
Gusenbauer, M. (2022). Search where you will find most: Comparing the disciplinary coverage of 56 bibliographic databases. Scientometrics, 127, 2683–2745. https://doi.org/10.1007/s11192-022-04289-7 [direct download]

Remove filters

Use the function to remove all filters and preferences you selected.

Our testing

What is the logic behind your testing?

We defined a generic testing procedure that works across a diverse set of academic search systems - all with distinct coverages, functionalities, and features. Thus, while other testing methods would be available, we chose the best common denominator across a heterogenic landscape of databases. This way, we can test a substantially greater number of databases compared to already existing database overviews.

We test the functionalities of specific capabilities search systems have or claim to have. Here we follow a routine that is called "metamorphic testing". It is a way of testing hard-to-test systems such as artificial intelligence, or databases. A group of researchers titled their 2020 IEEE article "Metamorphic Testing: Testing the Untestable". Using this logic, we test databases and systems that do not provide access to their systems.

Metamorphic testing is always done from the perspective of the user. It investigates how well a system performs, not at some theoretical level, but in practice - how well can the user search with a system? Do the results add up? What are the limitations of certain functionalities?

What specific tests do you have?

A detailed list of our tests can be found here. It is important to keep in mind that each of our tests can be independently verified by yourself. You can always check whether you replicate the test result we received.

How do you select the databases you test?

We want to include as many search systems and databases as possible. We started with 70 databases and frequently add new ones.

To be consistent in our testing and to warrant comparability, we need to focus on certain types of databases. Therefore, we have specific selection criteria for the inclusion of databases:

Databases with mostly scholarly content, e.g., journal articles, conference papers, academic books. However, it can be difficult to determine whether a database is 'scholarly' and how much scholarly content it has.
Databases with a disciplinary focus on at least one of the 26 ASJC subjects. Very narrowly focused databases are excluded, e.g., databases on theatre studies, a sub-discipline of Arts and Humanities.
Mostly English content (as much as we would love to include non-English-focused databases, this is not possible due to our testing procedures)
At first, we focus on large databases with more than 1,000,000 records, yet we also include smaller ones if they are popular. Clinical trials databases need to cover at least 100,000 trials. Often, smaller databases will be included in the larger databases we cover anyway.
Databases that inform about query hit counts, i.e., the number of hits a keyword query retrieves.

We want to include a diverse set of databases. We deliberately include open and paywalled, newer, and established databases in the same comparison. This helps readers assess their options compared to the databases they might already know.

There are many new, innovative search systems out there. We would like to include them all in our comparison. Yet, we need to stick to the requirements defined above. If you know a system we still miss, please contact us, and we will happily review it.

How do you determine subject coverage?

Subject coverage is based on the All Science Journal Classification used by Scopus. We chose this classification system due to its coverage of all science disciplines and some other reasons explained in a recent publication that outlines in detail how subject classification works. The intermediate level of granularity consists of 26 subjects (not counting 'multidisciplinary') to determine the focus of each database.

How do you determine keyword coverage?

Keyword coverage tells you the number of records that include the keyword in the title. Some values are computed as they do not allow determining coverage based on title-only searches.

How did you choose the keywords you use?

We used the number of search results for specific keywords to determine the subject coverage of databases. For each of the 26 subjects, we chose 14 keywords that were identified as most representative of the subject. We established representativeness by selecting keywords that were most prevalent in the titles of articles in the focussed subject while being least prevalent in article titles of other subjects. This way, the keywords were determined based on discriminant validity rather than chance, popularity, or frequency. Following our approach, we did not include terms such as 'evolution' that may have different meanings in different disciplines. Rather, we chose keywords such as 'boson' that were almost exclusively used in Physics, for example.

The only keyword we included that does not follow this logic is 'covid-19', which we included as a multi-disciplinary keyword of great relevance. Thus, users can determine the prevalence of 'covid-19' in scholarly databases we analyze.

Detailed information on our methodology can be found here: Gusenbauer, M. (2022). Search where you will find most: Comparing the disciplinary coverage of 56 bibliographic databases. Scientometrics, 127, 2683–2745. https://doi.org/10.1007/s11192-022-04289-7

How frequently do you update your data?

We have periodic intervals for automatic tests. The intervals are different for each test and can be weeks or months. As many search systems do not change frequently, update intervals of several months are not an issue. In general, databases update only gradually with steadily increasing coverage counts, making coverage assessments valid over long periods. If you find a system has updated and this is not yet reflected in our data, don't hesitate to get in touch with us.

What are the limitations of your testing?

As most (proprietary) system providers do not allow direct access to databases, metamorphic testing is the next best thing to independently verify the workings of their claimed functionalities.

Metamorphic testing cannot validate functionality - we can only show whether it is plausible to assume that a system works concerning functionalities the test covers.

This means it is plausible to assume that a system provides some functionality if it has passed our tests. Yet, we can never be fully sure it fails in some other circumstances our tests do not cover. We try to gear our metamorphic tests, so they are sensitive to system flaws, yet we cannot warrant that a system actually always works. In this regard, Search Smart is, however, a huge step forward from the situation before - where we had no systematic testing and monitoring of the systems millions of researchers build their research on.

About

What is the story behind Search Smart?

During my Ph.D. I felt the pain of not knowing whether I was using the best search options or not.

Search Smart is the tool I wish I had ten years ago when I started figuring out how to search (systematically). Identifying all literature on an ambiguously defined and labeled topic was no easy task. Armed with a search string of well over a thousand words, I quickly learned about different search systems' limitations. While some systems computed my queries, others did not. I learned the hard way how each one was different.

Search Smart is the product of a curiosity to understand and systematize academic search options. Since then, the goal has been to improve searching with increased transparency and better guidance.

"When content is abundant, content curation is an art." (Zarrin)

What is Search Smart's mission?

Like explorers must select their most appropriate telescope, explorers of the web must select their most appropriate search system. These lenses will direct their views and determine the things they will encounter.

Search Smart's mission is uncomplicated guidance of academics to find their best search options. Through meaningful and comprehensive testing of academic search systems that allow users to identify their best options with a few clicks.

Often, users are unaware of the great discrepancies in performance and functionalities between systems. Users often use what is popular among peers without being aware of alternatives. We want to make database selection more explicit and dynamic so users confidently know they search with the best option at hand.

What is Search Smart's vision?

Search Smart's vision is to foster educated researchers that use the best search options available. Heightened user expectations and a transparent search system landscape boost the continuous improvement of search systems that compete for being most fit-for-purpose.

We want to employ increasingly thorough testing to give an increasingly comprehensive overview of the landscape of scholarly databases. Search Smart should become an entity to stir up healthy competition among search providers, to make researchers demand more from search systems, and providers to better understand what functionalities are important to researchers for the pursuit of 'good science'.

Other

What do you mean by... ?

Here is a taxonomy of the most important terms we use at Search Smart:

Basket of keywords (BOK): Novel analytic method based on specific keyword queries allowing the assessment of subject coverage of a large number of bibliographic databases in academia.
Coverage: The prevalence of certain record types on a database.
Database: The underlying (bibliographic) database accessible via a search system.
Functionalities: The capabilities a search system offers users to search, filter, retrieve and manipulate search results.
Preset: Pre-defined filter setting that selects multiple filters at once.
Query hit count (QHC): Number of hits for a specific keyword query.
Records: Any type of file hosted on a database.
Search provider: The organization that operates the search system providing access to one or multiple databases.
Search system: The system that assesses a database. Can be a search engine, an aggregator, or some other type.

Are you affiliated with a search provider?

No, we are neither affiliated nor do we get any compensation for listing databases.

As a search provider, how can I access the information illustrated?

The tests Search Smart performs and the results it illustrates are collected independently of search system providers. Thus, providers cannot directly access or manipulate test results.

Nevertheless, providers can help provide detailed information on the workings of the underlying search mechanisms and the database they access. This helps Search Smart to most accurately illustrate the characteristics of the search systems. Please do not hesitate to contact us, particularly if you feel certain information may be inaccurate. We will look into any inconsistencies so we make sure to capture database and search interface characteristics as accurately as possible.

If you like to see a different logo representing your database, please get in touch with us, and we are happy to change it.

Why is Search Smart not working properly in my browser?

We have tested Search Smart with the latest versions of modern browsers (Chrome, Safari, Firefox, Edge). If you use a different browser, old an older version, please consider updating, so Search Smart may run smoothly. If you identify errors using up-to-date browsers, please contact us.

What is your donation policy, how can I help?

Your donation makes a difference!

Every donation, whether it's €2, €8, or a recurring amount, helps us cover daily operational costs, enhance our services, and conduct rigorous testing. To get the project started we have received funding from the Austrian Science Fund (https://www.fwf.ac.at/en/). Operations are funded by a lot of self-motivation, sweat, and private commitments. To support the ongoing enhancement and delivery of our services, we welcome donations from individuals and organizations. However, due to substantial administrative costs, donors are unable to designate specific uses for their donations.

Integrity and independence.

We are committed to maintaining the integrity and independence of our services, ensuring that financial contributions do not influence the impartiality of our database and search system representations. Our transparent, evidence-based evaluations are replicable and verifiable by users, ensuring that all results are objective and free from external influence. We do not ask for compensation for listing databases at Search Smart.

Donations from database providers.

All donations from database providers will be publicly disclosed on this page. Additionally, donations of €1,000 or more undergo rigorous vetting to ensure their alignment with our principles. Only donations, which align with the mission and values of Search Smart, especially its academic freedom and integrity, will be accepted. Such donations will be accepted only if accompanied by written consent from donors, acknowledging that their contributions do not entail any returns and that the allocation of funds is at the sole discretion of Search Smart.

Supporting Search Smart in other ways.

You may also help us by directing us to improvements and errors, suggesting new, eligible databases to review (see our focus in 'the FAQs') or listing us on your website or library page. If you are interested in individualized collaborations or providing long-term support please contact us directly at hi@searchsmart.org.