We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.DB

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Databases

Title: Can Deep Neural Networks Predict Data Correlations from Column Names?

Abstract: For humans, it is often possible to predict data correlations from column names. We conduct experiments to find out whether deep neural networks can learn to do the same. If so, e.g., it would open up the possibility of tuning tools that use NLP analysis on schema elements to prioritize their efforts for correlation detection.
We analyze correlations for around 120,000 column pairs, taken from around 4,000 data sets. We try to predict correlations, based on column names alone. For predictions, we exploit pre-trained language models, based on the recently proposed Transformer architecture. We consider different types of correlations, multiple prediction methods, and various prediction scenarios. We study the impact of factors such as column name length or the amount of training data on prediction accuracy. Altogether, we find that deep neural networks can predict correlations with a relatively high accuracy in many scenarios (e.g., with an accuracy of 95% for long column names).
Subjects: Databases (cs.DB); Computation and Language (cs.CL)
Cite as: arXiv:2107.04553 [cs.DB]
  (or arXiv:2107.04553v1 [cs.DB] for this version)

Submission history

From: Immanuel Trummer Mr. [view email]
[v1] Fri, 9 Jul 2021 17:11:54 GMT (839kb,D)

Link back to: arXiv, form interface, contact.