Abstract

This research aims to augment human cognition through the advancement and automation of mindmapping technologies, which could later support human creativity and virtual collaboration. Mindmapping is a visual brainstorming technique that allows problem solvers to utilize the human brain's ability to retrieve knowledge through similarity and association. While it is a powerful tool to generate concepts in any phase of s or design, the content of mindmaps is usually manually generated while listening or conversing and generating ideas, requiring a high cognitive load. This work introduces the development of a speech-driven automated mindmapping technology, called Speech2Mindmap. The specifics of the Speech2Mindmap algorithm are detailed, along with two case studies that serve to test its accuracy in comparison to human-generated mindmaps, using audio recorded speech data as input. In the first case study, the Speech2Mindmap algorithm was evaluated on how well it represents manually generated human mindmapping output. The second case study evaluated the reliability of the Speech2Mindmap algorithm and examined the best performing methods and conditions to achieve the greatest similarity to human-generated mindmaps. This research demonstrates that the Speech2Mindmap algorithm is capable of representing manually generated human mindmapping output and found the best performing methods and conditions to generate a mindmap that is 80% similar, on average, to human-generated mindmaps.

References

1.
Cross
,
N.
, and
Roy
,
R.
,
1989
,
Engineering Design Methods
,
Wiley
,
New York
.
2.
Thompson
,
G.
, and
Lordan
,
M.
,
1999
, “
A Review of Creativity Principles Applied to Engineering Design
,”
Proc. Inst. Mech. Eng., Part E
,
213
(
1
), pp.
17
31
.
3.
Pugh
,
S.
,
1991
,
Total Design: Integrated Methods for Successful Product Engineering
,
Addison-Wesley
,
Reading, MA
.
4.
Buzan
,
T.
, and
Buzan
,
B.
,
1993
,
The Mind Map Book
,
BBC Books
,
London
.
5.
Shneiderman
,
B.
,
2003
, “The Eyes Have it: A Task by Data Type Taxonomy for Information Visualizations,”
The Craft of Information Visualization
,
Elsevier
,
New York
, pp.
364
371
.
6.
Chen
,
J.
,
2008
, “
The Using of Mind Map in Concept Design
,”
9th International Conference on Computer-Aided Industrial Design and Conceptual Design (CAID/CD)
,
Kunming, China
,
Nov. 22–25
, IEEE, pp.
1034
1037
.
7.
Elhoseiny
,
M.
, and
Elgammal
,
A.
,
2016
, “
Text to Multi-Level MindMaps
,”
Multimed. Tools. Appl.
,
75
(
8
), pp.
4217
4244
.
8.
Lin
,
H.
, and
Faste
,
H.
,
2011
, “
Digital Mind Mapping: Innovations for Real-Time Collaborative Thinking
,”
CHI'11 Extended Abstracts on Human Factors in Computing Systems
,
Vancouver, BC, Canada
,
May 7–12
, ACM, pp.
2137
2142
.
9.
Otto
,
K. N.
,
2003
,
Product Design: Techniques in Reverse Engineering and New Product Development
,
Prentice Hall
,
Upper Saddle River, NJ
.
10.
Osborn
,
A.
,
1957
,
Applied Imagination
,
Scribner
,
New York, NY
.
11.
Elhoseiny
,
M.
, and
Elgammal
,
A.
,
2012
, “
English2mindmap: An Automated System for Mindmap Generation From English Text
,”
2012 IEEE International Symposium on Multimedia (ISM)
,
Irvine, CA
,
Dec. 10–12
, IEEE, pp.
326
331
.
12.
Kudelić
,
R.
,
Konecki
,
M.
, and
Maleković
,
M.
,
2011
, “
Mind Map Generator Software Model With Text Mining Algorithm
,”
Proceedings of the ITI 2011 33rd International Conference on Information Technology Interfaces (ITI)
,
Cavtat, Croatia
,
June 27–30
, IEEE, pp.
487
494
.
13.
Kudelić
,
R.
,
Maleković
,
M.
, and
Lovrenčić
,
A.
,
2012
, “
Mind Map Generator Software
,”
2012 IEEE International Conference on Computer Science and Automation Engineering (CSAE)
,
Zhangjiajie, China
,
May 25–27
, Vol. 3, IEEE, pp.
123
127
.
14.
Gonçalves
,
M.
,
Cardoso
,
C.
, and
Badke-Schaub
,
P.
,
2014
, “
What Inspires Designers? Preferences on Inspirational Approaches During Idea Generation
,”
Des. Stud.
,
35
(
1
), pp.
29
53
.
15.
Beel
,
J.
, and
Langer
,
S.
,
2011
, “
An Exploratory Analysis of Mind Maps
,”
Proceedings of the 11th ACM Symposium on Document Engineering
,
Mountain View, CA
,
Sept. 19–22
, ACM, pp.
81
84
.
16.
Tergan
,
S.-O.
,
2005
, “Digital Concept Maps for Managing Knowledge and Information,”
Knowledge and Information Visualization
,
Springer
,
New York
, pp.
185
204
.
17.
Redifer
,
J. L.
,
Bae
,
C. L.
, and
Zhao
,
Q.
, “
Self-Efficacy and Performance Feedback: Impacts on Cognitive Load During Creative Thinking
,”
Learn. Instr.
,
71
, p.
101395
.
18.
Thinkmap Inc.
ThinkMap
.”
Thinkmap Inc.
http://thinkmap.com/
19.
Speer
,
R.
,
Chin
,
J.
, and
Havasi
,
C.
,
2017
, “
Conceptnet 5.5: An Open Multilingual Graph of General Knowledge
,”
31st AAAI Conference on Artificial Intelligence
,
San Francisco, CA
,
Feb. 4–10
.
20.
Chen
,
T.-J.
, and
Krishnamurthy
,
V. R.
,
2020
, “
Investigating a Mixed-Initiative Workflow for Digital Mind-Mapping
,”
ASME J. Mech. Des.
,
142
(
10
), p.
101404
.
21.
Ang
,
S.
,
Rzadca
,
K.
, and
Datta
,
A.
,
2010
, “
SharedMind: A Tool for Collaborative Mind-Mapping
,”
2010 IEEE International Conference on Multimedia and Expo (ICME)
,
Singapore
,
July 19–23
, IEEE, pp.
1154
1155
.
22.
Shih
,
P. C.
,
Nguyen
,
D. H.
,
Hirano
,
S. H.
,
Redmiles
,
D. F.
, and
Hayes
,
G. R.
,
2009
, “
GroupMind: Supporting Idea Generation Through a Collaborative Mind-Mapping Tool
,”
Proceedings of the ACM 2009 International Conference on Supporting Group Work
,
Sanibel Island, FL
,
May 10–13
, ACM, pp.
139
148
.
23.
Hollingshead
,
A. B.
,
McGrath
,
J. E.
, and
O'Connor
,
K. M.
,
1993
, “
Group Task Performance and Communication Technology: A Longitudinal Study of Computer-Mediated Versus Face-to-Face Work Groups
,”
Small Group Res.
,
24
(
3
), pp.
307
333
.
24.
McLeod
,
P. L.
,
1992
, “
An Assessment of the Experimental Literature on Electronic Support of Group Work: Results of a Meta-Analysis
,”
Hum. Comput, Int.
,
7
(
3
), pp.
257
280
.
25.
Abdeen
,
M.
,
El-Sahan
,
R.
,
Ismaeil
,
A.
,
El-Harouny
,
S.
,
Shalaby
,
M.
, and
Yagoub
,
M.
,
2009
, “
Direct Automatic Generation of Mind Maps From Text With M 2 Gen
,”
2009 IEEE Toronto International Conference Science and Technology for Humanity (TIC-STH)
,
Toronto, ON, Canada
,
Sept. 26–27
, IEEE, pp.
95
99
.
26.
Hartmann
,
K. D.
,
2013
, “
The Power of Speech: Using Voice-to-Text Apps
,”
OT Pract.
,
18
(
14
), pp.
18
19
.
27.
Zhang
,
C.
,
2008
, “
Automatic Keyword Extraction From Documents Using Conditional Random Fields
,”
J. Comput. Inf. Syst.
,
4
(
3
), pp.
1169
1180
.
28.
Chen
,
P. I.
, and
Lin
,
S. J.
,
2010
, “
Automatic Keyword Prediction Using Google Similarity Distance
,”
Expert Syst. Appl.
,
37
(
3
), pp.
1928
1938
.
29.
Beliga
,
S.
,
Meštrović
,
A.
, and
Martinčić-Ipšić
,
S.
,
2015
, “
An Overview of Graph-Based Keyword Extraction Methods and Approaches
,”
J. Inf. Organ. Sci.
,
39
(
1
), pp.
1
20
.
30.
Cheong
,
H.
,
Li
,
W.
,
Cheung
,
A.
,
Nogueira
,
A.
, and
Iorio
,
F.
,
2017
, “
Automated Extraction of Function Knowledge From Text
,”
ASME J. Mech. Des.
,
139
(
11
), p.
111407
.
31.
Shi
,
F.
,
Chen
,
L.
,
Han
,
J.
, and
Childs
,
P.
,
2017
, “
A Data-Driven Text Mining and Semantic Network Analysis for Design Information Retrieval
,”
ASME J. Mech. Des.
,
139
(
11
), p.
111402
.
32.
Song
,
B.
,
Luo
,
J.
, and
Wood
,
K.
,
2019
, “
Data-Driven Platform Design: Patent Data and Function Network Analysis
,”
ASME J. Mech. Des.
,
141
(
2
), p.
021101
.
33.
Luo
,
J.
,
Yan
,
B.
, and
Wood
,
K.
,
2017
, “
InnoGPS for Data-Driven Exploration of Design Opportunities and Directions: The Case of Google Driverless car Project
,”
ASME J. Mech. Des.
,
139
(
11
), p.
111416
.
34.
Fu
,
K.
,
Cagan
,
J.
,
Kotovsky
,
K.
, and
Wood
,
K.
,
2013
, “
Discovering Structure in Design Databases Through Functional and Surface Based Mapping
,”
ASME J. Mech. Des.
,
135
(
3
), p.
031006
.
35.
Fu
,
K.
,
Chan
,
J.
,
Cagan
,
J.
,
Kotovsky
,
K.
,
Schunn
,
C.
, and
Wood
,
K.
,
2013
, “
The Meaning of “Near” and “Far”: the Impact of Structuring Design Databases and the Effect of Distance of Analogy on Design Output
,”
ASME J. Mech. Des.
,
135
(
2
), p.
021007
.
36.
Steyvers
,
M.
, and
Tenenbaum
,
J. B.
,
2005
, “
The Large-Scale Structure of Semantic Networks: Statistical Analyses and a Model of Semantic Growth
,”
Cogn. Sci.
,
29
(
1
), pp.
41
78
.
37.
Mihalcea
,
R.
, and
Radev
,
D.
,
2011
,
Graph-Based Natural Language Processing and Information Retrieval
,
Cambridge University Press
,
Cambridge, UK
.
38.
Solé
,
R. V.
,
Corominas-Murtra
,
B.
,
Valverde
,
S.
, and
Steels
,
L. J. C.
,
2010
, “
Language Networks: Their Structure, Function, and Evolution
,”
Complexity
,
15
(
6
), pp.
20
26
.
39.
Sonawane
,
S. S.
, and
Kulkarni
,
P. A.
,
2014
, “
Graph Based Representation and Analysis of Text Document: A Survey of Techniques
,”
Int. J. Comput. Appl.
,
96
(
19
), pp.
1
8
.
40.
Li
,
X.
,
Chen
,
C.-H.
,
Zheng
,
P.
,
Wang
,
Z.
,
Jiang
,
Z.
, and
Jiang
,
Z.
,
2020
, “
A Knowledge Graph-Aided Concept–Knowledge Approach for Evolutionary Smart Product–Service System Development
,”
ASME J. Mech. Des.
,
142
(
10
), p.
101403
.
41.
Lim
,
S. Y. C.
,
Camburn
,
B. A.
,
Moreno
,
D.
,
Huang
,
Z.
, and
Wood
,
K.
,
2016
, “
Design Concept Structures in Massive Group Ideation
,”
International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Charlotte, NC
,
Aug. 21–24
, Vol. 50190, American Society of Mechanical Engineers, p.
V007T06A006
.
42.
He
,
Y.
,
Camburn
,
B.
,
Liu
,
H.
,
Luo
,
J.
,
Yang
,
M.
, and
Wood
,
K.
,
2019
, “
Mining and Representing the Concept Space of Existing Ideas for Directed Ideation
,”
ASME J. Mech. Des.
,
141
(
12
), p.
121101
.
43.
Mihalcea
,
R.
, and
Tarau
,
P.
,
2004
, “
Textrank: Bringing Order Into Text
,”
Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing
,
Barcelona, Spain
,
July 25–26
.
44.
Brin
,
S.
, and
Page
,
L.
,
1998
, “
The Anatomy of a Large-Scale Hypertextual web Search Engine
,”
J. Comput. Networks ISDN Syst.
,
30
(
1–7
), pp.
107
117
.
45.
Jamieson
,
P.
,
2012
, “
Using Modern Graph Analysis Techniques on Mind Maps to Help Quantify Learning
,”
IEEE 2012 Frontiers in Education (FIE) Conference
,
Seattle, WA
,
Oct. 3–6
, IEEE, pp.
1
6
.
46.
Franklin
,
A.
,
Li
,
T.
,
Jamieson
,
P.
,
Semlak
,
J.
, and
Vanderbush
,
W.
,
2015
, “
Evaluating Metrics for Automatic Mind Map Assessment in Various Classes
,”
2015 IEEE Frontiers in Education (FIE) Conference
,
Camino Real El Paso, El Paso, TX
,
Oct. 21–24
, IEEE, pp.
1
8
.
You do not currently have access to this content.