1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
|
====================
Standard C++ Modules
====================
.. contents::
:local:
Introduction
============
The term ``modules`` has a lot of meanings. For the users of Clang, modules may
refer to ``Objective-C Modules``, ``Clang C++ Modules`` (or ``Clang Header Modules``,
etc.) or ``Standard C++ Modules``. The implementation of all these kinds of modules in Clang
has a lot of shared code, but from the perspective of users, their semantics and
command line interfaces are very different. This document focuses on
an introduction of how to use standard C++ modules in Clang.
There is already a detailed document about `Clang modules <Modules.html>`_, it
should be helpful to read `Clang modules <Modules.html>`_ if you want to know
more about the general idea of modules. Since standard C++ modules have different semantics
(and work flows) from `Clang modules`, this page describes the background and use of
Clang with standard C++ modules.
Modules exist in two forms in the C++ Language Specification. They can refer to
either "Named Modules" or to "Header Units". This document covers both forms.
Standard C++ Named modules
==========================
This document was intended to be a manual first and foremost, however, we consider it helpful to
introduce some language background here for readers who are not familiar with
the new language feature. This document is not intended to be a language
tutorial; it will only introduce necessary concepts about the
structure and building of the project.
Background and terminology
--------------------------
Modules
~~~~~~~
In this document, the term ``Modules``/``modules`` refers to standard C++ modules
feature if it is not decorated by ``Clang``.
Clang Modules
~~~~~~~~~~~~~
In this document, the term ``Clang Modules``/``Clang modules`` refer to Clang
c++ modules extension. These are also known as ``Clang header modules``,
``Clang module map modules`` or ``Clang c++ modules``.
Module and module unit
~~~~~~~~~~~~~~~~~~~~~~
A module consists of one or more module units. A module unit is a special
translation unit. Every module unit must have a module declaration. The syntax
of the module declaration is:
.. code-block:: c++
[export] module module_name[:partition_name];
Terms enclosed in ``[]`` are optional. The syntax of ``module_name`` and ``partition_name``
in regex form corresponds to ``[a-zA-Z_][a-zA-Z_0-9\.]*``. In particular, a literal dot ``.``
in the name has no semantic meaning (e.g. implying a hierarchy).
In this document, module units are classified into:
* Primary module interface unit.
* Module implementation unit.
* Module interface partition unit.
* Internal module partition unit.
A primary module interface unit is a module unit whose module declaration is
``export module module_name;``. The ``module_name`` here denotes the name of the
module. A module should have one and only one primary module interface unit.
A module implementation unit is a module unit whose module declaration is
``module module_name;``. A module could have multiple module implementation
units with the same declaration.
A module interface partition unit is a module unit whose module declaration is
``export module module_name:partition_name;``. The ``partition_name`` should be
unique within any given module.
An internal module partition unit is a module unit whose module declaration
is ``module module_name:partition_name;``. The ``partition_name`` should be
unique within any given module.
In this document, we use the following umbrella terms:
* A ``module interface unit`` refers to either a ``primary module interface unit``
or a ``module interface partition unit``.
* An ``importable module unit`` refers to either a ``module interface unit``
or a ``internal module partition unit``.
* A ``module partition unit`` refers to either a ``module interface partition unit``
or a ``internal module partition unit``.
Built Module Interface file
~~~~~~~~~~~~~~~~~~~~~~~~~~~
A ``Built Module Interface file`` stands for the precompiled result of an importable module unit.
It is also called the acronym ``BMI`` generally.
Global module fragment
~~~~~~~~~~~~~~~~~~~~~~
In a module unit, the section from ``module;`` to the module declaration is called the global module fragment.
How to build projects using modules
-----------------------------------
Quick Start
~~~~~~~~~~~
Let's see a "hello world" example that uses modules.
.. code-block:: c++
// Hello.cppm
module;
#include <iostream>
export module Hello;
export void hello() {
std::cout << "Hello World!\n";
}
// use.cpp
import Hello;
int main() {
hello();
return 0;
}
Then we type:
.. code-block:: console
$ clang++ -std=c++20 Hello.cppm --precompile -o Hello.pcm
$ clang++ -std=c++20 use.cpp -fmodule-file=Hello=Hello.pcm Hello.pcm -o Hello.out
$ ./Hello.out
Hello World!
In this example, we make and use a simple module ``Hello`` which contains only a
primary module interface unit ``Hello.cppm``.
Then let's see a little bit more complex "hello world" example which uses the 4 kinds of module units.
.. code-block:: c++
// M.cppm
export module M;
export import :interface_part;
import :impl_part;
export void Hello();
// interface_part.cppm
export module M:interface_part;
export void World();
// impl_part.cppm
module;
#include <iostream>
#include <string>
module M:impl_part;
import :interface_part;
std::string W = "World.";
void World() {
std::cout << W << std::endl;
}
// Impl.cpp
module;
#include <iostream>
module M;
void Hello() {
std::cout << "Hello ";
}
// User.cpp
import M;
int main() {
Hello();
World();
return 0;
}
Then we are able to compile the example by the following command:
.. code-block:: console
# Precompiling the module
$ clang++ -std=c++20 interface_part.cppm --precompile -o M-interface_part.pcm
$ clang++ -std=c++20 impl_part.cppm --precompile -fprebuilt-module-path=. -o M-impl_part.pcm
$ clang++ -std=c++20 M.cppm --precompile -fprebuilt-module-path=. -o M.pcm
$ clang++ -std=c++20 Impl.cpp -fprebuilt-module-path=. -c -o Impl.o
# Compiling the user
$ clang++ -std=c++20 User.cpp -fprebuilt-module-path=. -c -o User.o
# Compiling the module and linking it together
$ clang++ -std=c++20 M-interface_part.pcm -fprebuilt-module-path=. -c -o M-interface_part.o
$ clang++ -std=c++20 M-impl_part.pcm -fprebuilt-module-path=. -c -o M-impl_part.o
$ clang++ -std=c++20 M.pcm -fprebuilt-module-path=. -c -o M.o
$ clang++ User.o M-interface_part.o M-impl_part.o M.o Impl.o -o a.out
We explain the options in the following sections.
How to enable standard C++ modules
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Currently, standard C++ modules are enabled automatically
if the language standard is ``-std=c++20`` or newer.
How to produce a BMI
~~~~~~~~~~~~~~~~~~~~
We can generate a BMI for an importable module unit by either ``--precompile``
or ``-fmodule-output`` flags.
The ``--precompile`` option generates the BMI as the output of the compilation and the output path
can be specified using the ``-o`` option.
The ``-fmodule-output`` option generates the BMI as a by-product of the compilation.
If ``-fmodule-output=`` is specified, the BMI will be emitted the specified location. Then if
``-fmodule-output`` and ``-c`` are specified, the BMI will be emitted in the directory of the
output file with the name of the input file with the new extension ``.pcm``. Otherwise, the BMI
will be emitted in the working directory with the name of the input file with the new extension
``.pcm``.
The style to generate BMIs by ``--precompile`` is called two-phase compilation since it takes
2 steps to compile a source file to an object file. The style to generate BMIs by ``-fmodule-output``
is called one-phase compilation respectively. The one-phase compilation model is simpler
for build systems to implement and the two-phase compilation has the potential to compile faster due
to higher parallelism. As an example, if there are two module units A and B, and B depends on A, the
one-phase compilation model would need to compile them serially, whereas the two-phase compilation
model may be able to compile them simultaneously if the compilation from A.pcm to A.o takes a long
time.
File name requirement
~~~~~~~~~~~~~~~~~~~~~
The file name of an ``importable module unit`` should end with ``.cppm``
(or ``.ccm``, ``.cxxm``, ``.c++m``). The file name of a ``module implementation unit``
should end with ``.cpp`` (or ``.cc``, ``.cxx``, ``.c++``).
The file name of BMIs should end with ``.pcm``.
The file name of the BMI of a ``primary module interface unit`` should be ``module_name.pcm``.
The file name of BMIs of ``module partition unit`` should be ``module_name-partition_name.pcm``.
If the file names use different extensions, Clang may fail to build the module.
For example, if the filename of an ``importable module unit`` ends with ``.cpp`` instead of ``.cppm``,
then we can't generate a BMI for the ``importable module unit`` by ``--precompile`` option
since ``--precompile`` option now would only run preprocessor, which is equal to `-E` now.
If we want the filename of an ``importable module unit`` ends with other suffixes instead of ``.cppm``,
we could put ``-x c++-module`` in front of the file. For example,
.. code-block:: c++
// Hello.cpp
module;
#include <iostream>
export module Hello;
export void hello() {
std::cout << "Hello World!\n";
}
// use.cpp
import Hello;
int main() {
hello();
return 0;
}
Now the filename of the ``module interface`` ends with ``.cpp`` instead of ``.cppm``,
we can't compile them by the original command lines. But we are still able to do it by:
.. code-block:: console
$ clang++ -std=c++20 -x c++-module Hello.cpp --precompile -o Hello.pcm
$ clang++ -std=c++20 use.cpp -fprebuilt-module-path=. Hello.pcm -o Hello.out
$ ./Hello.out
Hello World!
Module name requirement
~~~~~~~~~~~~~~~~~~~~~~~
[module.unit]p1 says:
.. code-block:: text
All module-names either beginning with an identifier consisting of std followed by zero
or more digits or containing a reserved identifier ([lex.name]) are reserved and shall not
be specified in a module-declaration; no diagnostic is required. If any identifier in a reserved
module-name is a reserved identifier, the module name is reserved for use by C++ implementations;
otherwise it is reserved for future standardization.
So all of the following name is not valid by default:
.. code-block:: text
std
std1
std.foo
__test
// and so on ...
If you still want to use the reserved module names for any reason, use
``-Wno-reserved-module-identifier`` to suppress the warning.
How to specify the dependent BMIs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
There are 3 methods to specify the dependent BMIs:
* (1) ``-fprebuilt-module-path=<path/to/directory>``.
* (2) ``-fmodule-file=<path/to/BMI>`` (Deprecated).
* (3) ``-fmodule-file=<module-name>=<path/to/BMI>``.
The option ``-fprebuilt-module-path`` tells the compiler the path where to search for dependent BMIs.
It may be used multiple times just like ``-I`` for specifying paths for header files. The look up rule here is:
* (1) When we import module M. The compiler would look up M.pcm in the directories specified
by ``-fprebuilt-module-path``.
* (2) When we import partition module unit M:P. The compiler would look up M-P.pcm in the
directories specified by ``-fprebuilt-module-path``.
The option ``-fmodule-file=<path/to/BMI>`` tells the compiler to load the specified BMI directly.
The option ``-fmodule-file=<module-name>=<path/to/BMI>`` tells the compiler to load the specified BMI
for the module specified by ``<module-name>`` when necessary. The main difference is that
``-fmodule-file=<path/to/BMI>`` will load the BMI eagerly, whereas
``-fmodule-file=<module-name>=<path/to/BMI>`` will only load the BMI lazily, which is similar
with ``-fprebuilt-module-path``. The option ``-fmodule-file=<path/to/BMI>`` for named modules is deprecated
and is planning to be removed in future versions.
In case all ``-fprebuilt-module-path=<path/to/directory>``, ``-fmodule-file=<path/to/BMI>`` and
``-fmodule-file=<module-name>=<path/to/BMI>`` exist, the ``-fmodule-file=<path/to/BMI>`` option
takes highest precedence and ``-fmodule-file=<module-name>=<path/to/BMI>`` will take the second
highest precedence.
We need to specify all the dependent (directly and indirectly) BMIs.
See https://github.com/llvm/llvm-project/issues/62707 for detail.
When we compile a ``module implementation unit``, we must specify the BMI of the corresponding
``primary module interface unit``.
Since the language specification says a module implementation unit implicitly imports
the primary module interface unit.
[module.unit]p8
A module-declaration that contains neither an export-keyword nor a module-partition implicitly
imports the primary module interface unit of the module as if by a module-import-declaration.
All of the 3 options ``-fprebuilt-module-path=<path/to/directory>``, ``-fmodule-file=<path/to/BMI>``
and ``-fmodule-file=<module-name>=<path/to/BMI>`` may occur multiple times.
For example, the command line to compile ``M.cppm`` in
the above example could be rewritten into:
.. code-block:: console
$ clang++ -std=c++20 M.cppm --precompile -fmodule-file=M:interface_part=M-interface_part.pcm -fmodule-file=M:impl_part=M-impl_part.pcm -o M.pcm
When there are multiple ``-fmodule-file=<module-name>=`` options for the same
``<module-name>``, the last ``-fmodule-file=<module-name>=`` will override the previous
``-fmodule-file=<module-name>=`` options.
``-fprebuilt-module-path`` is more convenient and ``-fmodule-file`` is faster since
it saves time for file lookup.
Remember that module units still have an object counterpart to the BMI
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It is easy to forget to compile BMIs at first since we may envision module interfaces like headers.
However, this is not true.
Module units are translation units. We need to compile them to object files
and link the object files like the example shows.
For example, the traditional compilation processes for headers are like:
.. code-block:: text
src1.cpp -+> clang++ src1.cpp --> src1.o ---,
hdr1.h --' +-> clang++ src1.o src2.o -> executable
hdr2.h --, |
src2.cpp -+> clang++ src2.cpp --> src2.o ---'
And the compilation process for module units are like:
.. code-block:: text
src1.cpp ----------------------------------------+> clang++ src1.cpp -------> src1.o -,
(header unit) hdr1.h -> clang++ hdr1.h ... -> hdr1.pcm --' +-> clang++ src1.o mod1.o src2.o -> executable
mod1.cppm -> clang++ mod1.cppm ... -> mod1.pcm --,--> clang++ mod1.pcm ... -> mod1.o -+
src2.cpp ----------------------------------------+> clang++ src2.cpp -------> src2.o -'
As the diagrams show, we need to compile the BMI from module units to object files and link the object files.
(But we can't do this for the BMI from header units. See the later section for the definition of header units)
If we want to create a module library, we can't just ship the BMIs in an archive.
We must compile these BMIs(``*.pcm``) into object files(``*.o``) and add those object files to the archive instead.
Consistency Requirement
~~~~~~~~~~~~~~~~~~~~~~~
If we envision modules as a cache to speed up compilation, then - as with other caching techniques -
it is important to keep cache consistency.
So **currently** Clang will do very strict check for consistency.
Options consistency
^^^^^^^^^^^^^^^^^^^
The language option of module units and their non-module-unit users should be consistent.
The following example is not allowed:
.. code-block:: c++
// M.cppm
export module M;
// Use.cpp
import M;
.. code-block:: console
$ clang++ -std=c++20 M.cppm --precompile -o M.pcm
$ clang++ -std=c++23 Use.cpp -fprebuilt-module-path=.
The compiler would reject the example due to the inconsistent language options.
Not all options are language options.
For example, the following example is allowed:
.. code-block:: console
$ clang++ -std=c++20 M.cppm --precompile -o M.pcm
# Inconsistent optimization level.
$ clang++ -std=c++20 -O3 Use.cpp -fprebuilt-module-path=.
# Inconsistent debugging level.
$ clang++ -std=c++20 -g Use.cpp -fprebuilt-module-path=.
Although the two examples have inconsistent optimization and debugging level, both of them are accepted.
Note that **currently** the compiler doesn't consider inconsistent macro definition a problem. For example:
.. code-block:: console
$ clang++ -std=c++20 M.cppm --precompile -o M.pcm
# Inconsistent optimization level.
$ clang++ -std=c++20 -O3 -DNDEBUG Use.cpp -fprebuilt-module-path=.
Currently Clang would accept the above example. But it may produce surprising results if the
debugging code depends on consistent use of ``NDEBUG`` also in other translation units.
Definitions consistency
^^^^^^^^^^^^^^^^^^^^^^^
The C++ language defines that same declarations in different translation units should have
the same definition, as known as ODR (One Definition Rule). Prior to modules, the translation
units don't dependent on each other and the compiler itself can't perform a strong
ODR violation check. With the introduction of modules, now the compiler have
the chance to perform ODR violations with language semantics across translation units.
However, in the practice, we found the existing ODR checking mechanism is not stable
enough. Many people suffers from the false positive ODR violation diagnostics, AKA,
the compiler are complaining two identical declarations have different definitions
incorrectly. Also the true positive ODR violations are rarely reported.
Also we learned that MSVC don't perform ODR check for declarations in the global module
fragment.
So in order to get better user experience, save the time checking ODR and keep consistent
behavior with MSVC, we disabled the ODR check for the declarations in the global module
fragment by default. Users who want more strict check can still use the
``-Xclang -fno-skip-odr-check-in-gmf`` flag to get the ODR check enabled. It is also
encouraged to report issues if users find false positive ODR violations or false negative ODR
violations with the flag enabled.
ABI Impacts
-----------
The declarations in a module unit which are not in the global module fragment have new linkage names.
For example,
.. code-block:: c++
export module M;
namespace NS {
export int foo();
}
The linkage name of ``NS::foo()`` would be ``_ZN2NSW1M3fooEv``.
This couldn't be demangled by previous versions of the debugger or demangler.
As of LLVM 15.x, users can utilize ``llvm-cxxfilt`` to demangle this:
.. code-block:: console
$ llvm-cxxfilt _ZN2NSW1M3fooEv
The result would be ``NS::foo@M()``, which reads as ``NS::foo()`` in module ``M``.
The ABI implies that we can't declare something in a module unit and define it in a non-module unit (or vice-versa),
as this would result in linking errors.
If we still want to implement declarations within the compatible ABI in module unit,
we can use the language-linkage specifier. Since the declarations in the language-linkage specifier
is attached to the global module fragments. For example:
.. code-block:: c++
export module M;
namespace NS {
export extern "C++" int foo();
}
Now the linkage name of ``NS::foo()`` will be ``_ZN2NS3fooEv``.
Performance Tips
----------------
Reduce duplications
~~~~~~~~~~~~~~~~~~~
While it is legal to have duplicated declarations in the global module fragments
of different module units, it is not free for clang to deal with the duplicated
declarations. In other word, for a translation unit, it will compile slower if the
translation unit itself and its importing module units contains a lot duplicated
declarations.
For example,
.. code-block:: c++
// M-partA.cppm
module;
#include "big.header.h"
export module M:partA;
...
// M-partB.cppm
module;
#include "big.header.h"
export module M:partB;
...
// other partitions
...
// M-partZ.cppm
module;
#include "big.header.h"
export module M:partZ;
...
// M.cppm
export module M;
export import :partA;
export import :partB;
...
export import :partZ;
// use.cpp
import M;
... // use declarations from module M.
When ``big.header.h`` is big enough and there are a lot of partitions,
the compilation of ``use.cpp`` may be slower than
the following style significantly:
.. code-block:: c++
module;
#include "big.header.h"
export module m:big.header.wrapper;
export ... // export the needed declarations
// M-partA.cppm
export module M:partA;
import :big.header.wrapper;
...
// M-partB.cppm
export module M:partB;
import :big.header.wrapper;
...
// other partitions
...
// M-partZ.cppm
export module M:partZ;
import :big.header.wrapper;
...
// M.cppm
export module M;
export import :partA;
export import :partB;
...
export import :partZ;
// use.cpp
import M;
... // use declarations from module M.
The key part of the tip is to reduce the duplications from the text includes.
Ideas for converting to modules
-------------------------------
For new libraries, we encourage them to use modules completely from day one if possible.
This will be pretty helpful to make the whole ecosystems to get ready.
For many existing libraries, it may be a breaking change to refactor themselves
into modules completely. So that many existing libraries need to provide headers and module
interfaces for a while to not break existing users.
Here we provide some ideas to ease the transition process for existing libraries.
**Note that the this section is only about helping ideas instead of requirement from clang**.
Let's start with the case that there is no dependency or no dependent libraries providing
modules for your library.
ABI non-breaking styles
~~~~~~~~~~~~~~~~~~~~~~~
export-using style
^^^^^^^^^^^^^^^^^^
.. code-block:: c++
module;
#include "header_1.h"
#include "header_2.h"
...
#include "header_n.h"
export module your_library;
export namespace your_namespace {
using decl_1;
using decl_2;
...
using decl_n;
}
As the example shows, you need to include all the headers containing declarations needs
to be exported and `using` such declarations in an `export` block. Then, basically,
we're done.
export extern-C++ style
^^^^^^^^^^^^^^^^^^^^^^^
.. code-block:: c++
module;
#include "third_party/A/headers.h"
#include "third_party/B/headers.h"
...
#include "third_party/Z/headers.h"
export module your_library;
#define IN_MODULE_INTERFACE
extern "C++" {
#include "header_1.h"
#include "header_2.h"
...
#include "header_n.h"
}
Then in your headers (from ``header_1.h`` to ``header_n.h``), you need to define the macro:
.. code-block:: c++
#ifdef IN_MODULE_INTERFACE
#define EXPORT export
#else
#define EXPORT
#endif
And you should put ``EXPORT`` to the beginning of the declarations you want to export.
Also it is suggested to refactor your headers to include thirdparty headers conditionally:
.. code-block:: c++
#ifndef IN_MODULE_INTERFACE
#include "third_party/A/headers.h"
#endif
#include "header_x.h"
...
This may be helpful to get better diagnostic messages if you forgot to update your module
interface unit file during maintaining.
The reasoning for the practice is that the declarations in the language linkage are considered
to be attached to the global module. So the ABI of your library in the modular version
wouldn't change.
While this style looks not as convenient as the export-using style, it is easier to convert
to other styles.
ABI breaking style
~~~~~~~~~~~~~~~~~~
The term ``ABI breaking`` sounds terrifying generally. But you may want it here if you want
to force your users to introduce your library in a consistent way. E.g., they either include
your headers all the way or import your modules all the way.
The style prevents the users to include your headers and import your modules at the same time
in the same repo.
The pattern for ABI breaking style is similar with export extern-C++ style.
.. code-block:: c++
module;
#include "third_party/A/headers.h"
#include "third_party/B/headers.h"
...
#include "third_party/Z/headers.h"
export module your_library;
#define IN_MODULE_INTERFACE
#include "header_1.h"
#include "header_2.h"
...
#include "header_n.h"
#if the number of .cpp files in your project are small
module :private;
#include "source_1.cpp"
#include "source_2.cpp"
...
#include "source_n.cpp"
#else // the number of .cpp files in your project are a lot
// Using all the declarations from thirdparty libraries which are
// used in the .cpp files.
namespace third_party_namespace {
using third_party_decl_used_in_cpp_1;
using third_party_decl_used_in_cpp_2;
...
using third_party_decl_used_in_cpp_n;
}
#endif
(And add `EXPORT` and conditional include to the headers as suggested in the export
extern-C++ style section)
Remember that the ABI get changed and we need to compile our source files into the
new ABI format. This is the job of the additional part of the interface unit:
.. code-block:: c++
#if the number of .cpp files in your project are small
module :private;
#include "source_1.cpp"
#include "source_2.cpp"
...
#include "source_n.cpp"
#else // the number of .cpp files in your project are a lot
// Using all the declarations from thirdparty libraries which are
// used in the .cpp files.
namespace third_party_namespace {
using third_party_decl_used_in_cpp_1;
using third_party_decl_used_in_cpp_2;
...
using third_party_decl_used_in_cpp_n;
}
#endif
In case the number of your source files are small, we may put everything in the private
module fragment directly. (it is suggested to add conditional include to the source
files too). But it will make the compilation of the module interface unit to be slow
when the number of the source files are not small enough.
**Note that the private module fragment can only be in the primary module interface unit
and the primary module interface unit containing private module fragment should be the only
module unit of the corresponding module.**
In that case, you need to convert your source files (.cpp files) to module implementation units:
.. code-block:: c++
#ifndef IN_MODULE_INTERFACE
// List all the includes here.
#include "third_party/A/headers.h"
...
#include "header.h"
#endif
module your_library;
// Following off should be unchanged.
...
The module implementation unit will import the primary module implicitly.
We don't include any headers in the module implementation units
here since we want to avoid duplicated declarations between translation units.
This is the reason why we add non-exported using declarations from the third
party libraries in the primary module interface unit.
And if you provide your library as ``libyour_library.so``, you probably need to
provide a modular one ``libyour_library_modules.so`` since you changed the ABI.
What if there are headers only inclued by the source files
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The above practice may be problematic if there are headers only included by the source
files. If you're using private module fragment, you may solve the issue by including them
in the private module fragment. While it is OK to solve it by including the implementation
headers in the module purview if you're using implementation module units, it may be
suboptimal since the primary module interface units now containing entities not belongs
to the interface.
If you're a perfectionist, maybe you can improve it by introducing internal module partition unit.
The internal module partition unit is an importable module unit which is internal
to the module itself. The concept just meets the headers only included by the source files.
We don't show code snippet since it may be too verbose or not good or not general.
But it may not be too hard if you can understand the points of the section.
Providing a header to skip parsing redundant headers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It is a problem for clang to handle redeclarations between translation units.
Also there is a long standing issue in clang (`problematic include after import <https://github.com/llvm/llvm-project/issues/61465>`_).
But even if the issue get fixed in clang someday, the users may still get slower compilation speed
and larger BMI size. So it is suggested to not include headers after importing the corresponding
library.
However, it is not easy for users if your library are included by other dependencies.
So the users may have to write codes like:
.. code-block:: c++
#include "third_party/A.h" // #include "your_library/a_header.h"
import your_library;
or
.. code-block:: c++
import your_library;
#include "third_party/A.h" // #include "your_library/a_header.h"
For such cases, we suggest the libraries providing modules and the headers at the same time
to provide a header to skip parsing all the headers in your libraries. So the users can
import your library as the following style to skip redundant handling:
.. code-block:: c++
import your_library;
#include "your_library_imported.h"
#include "third_party/A.h" // #include "your_library/a_header.h" but got skipped
The implementation of ``your_library_imported.h`` can be a set of controlling macros or
an overall controlling macro if you're using `#pragma once`. So you can convert your
headers to:
.. code-block:: c++
#pragma once
#ifndef YOUR_LIBRARY_IMPORTED
...
#endif
Importing modules
~~~~~~~~~~~~~~~~~
When there are dependent libraries providing modules, we suggest you to import that in
your module.
Most of the existing libraries would fall into this catagory once the std module gets available.
All dependent libraries providing modules
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Life gets easier if all the dependent libraries providing modules.
You need to convert your headers to include thirdparty headers conditionally.
Then for export-using style:
.. code-block:: c++
module;
import modules_from_third_party;
#define IN_MODULE_INTERFACE
#include "header_1.h"
#include "header_2.h"
...
#include "header_n.h"
export module your_library;
export namespace your_namespace {
using decl_1;
using decl_2;
...
using decl_n;
}
For export extern-C++ style:
.. code-block:: c++
export module your_library;
import modules_from_third_party;
#define IN_MODULE_INTERFACE
extern "C++" {
#include "header_1.h"
#include "header_2.h"
...
#include "header_n.h"
}
For ABI breaking style,
.. code-block:: c++
export module your_library;
import modules_from_third_party;
#define IN_MODULE_INTERFACE
#include "header_1.h"
#include "header_2.h"
...
#include "header_n.h"
#if the number of .cpp files in your project are small
module :private;
#include "source_1.cpp"
#include "source_2.cpp"
...
#include "source_n.cpp"
#endif
We don't need the non-exported using declarations if we're using implementation module
units now. We can import thirdparty modules directly in the implementation module
units.
Partial dependent libraries providing modules
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In this case, we have to mix the use of ``include`` and ``import`` in the module of our
library. The key point here is still to remove duplicated declarations in translation
units as much as possible. If the imported modules provide headers to skip parsing their
headers, we should include that after the including. If the imported modules don't provide
the headers, we can make it ourselves if we still want to optimize it.
Known Problems
--------------
The following describes issues in the current implementation of modules.
Please see https://github.com/llvm/llvm-project/labels/clang%3Amodules for more issues
or file a new issue if you don't find an existing one.
If you're going to create a new issue for standard C++ modules,
please start the title with ``[C++20] [Modules]`` (or ``[C++23] [Modules]``, etc)
and add the label ``clang:modules`` (if you have permissions for that).
For higher level support for proposals, you could visit https://clang.llvm.org/cxx_status.html.
Including headers after import is problematic
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For example, the following example can be accept:
.. code-block:: c++
#include <iostream>
import foo; // assume module 'foo' contain the declarations from `<iostream>`
int main(int argc, char *argv[])
{
std::cout << "Test\n";
return 0;
}
but it will get rejected if we reverse the order of ``#include <iostream>`` and
``import foo;``:
.. code-block:: c++
import foo; // assume module 'foo' contain the declarations from `<iostream>`
#include <iostream>
int main(int argc, char *argv[])
{
std::cout << "Test\n";
return 0;
}
Both of the above examples should be accepted.
This is a limitation in the implementation. In the first example,
the compiler will see and parse <iostream> first then the compiler will see the import.
So the ODR Checking and declarations merging will happen in the deserializer.
In the second example, the compiler will see the import first and the include second.
As a result, the ODR Checking and declarations merging will happen in the semantic analyzer.
So there is divergence in the implementation path. It might be understandable that why
the orders matter here in the case.
(Note that "understandable" is different from "makes sense").
This is tracked in: https://github.com/llvm/llvm-project/issues/61465
Ignored PreferredName Attribute
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Due to a tricky problem, when Clang writes BMIs, Clang will ignore the ``preferred_name`` attribute, if any.
This implies that the ``preferred_name`` wouldn't show in debugger or dumping.
This is tracked in: https://github.com/llvm/llvm-project/issues/56490
Don't emit macros about module declaration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is covered by P1857R3. We mention it again here since users may abuse it before we implement it.
Someone may want to write code which could be compiled both by modules or non-modules.
A direct idea would be use macros like:
.. code-block:: c++
MODULE
IMPORT header_name
EXPORT_MODULE MODULE_NAME;
IMPORT header_name
EXPORT ...
So this file could be triggered like a module unit or a non-module unit depending on the definition
of some macros.
However, this kind of usage is forbidden by P1857R3 but we haven't implemented P1857R3 yet.
This means that is possible to write illegal modules code now, and obviously this will stop working
once P1857R3 is implemented.
A simple suggestion would be "Don't play macro tricks with module declarations".
This is tracked in: https://github.com/llvm/llvm-project/issues/56917
In consistent filename suffix requirement for importable module units
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Currently, clang requires the file name of an ``importable module unit`` should end with ``.cppm``
(or ``.ccm``, ``.cxxm``, ``.c++m``). However, the behavior is inconsistent with other compilers.
This is tracked in: https://github.com/llvm/llvm-project/issues/57416
clang-cl is not compatible with the standard C++ modules
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now we can't use the `/clang:-fmodule-file` or `/clang:-fprebuilt-module-path` to specify
the BMI within ``clang-cl.exe``.
This is tracked in: https://github.com/llvm/llvm-project/issues/64118
false positive ODR violation diagnostic due to using inconsistent qualified but the same type
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ODR violation is a pretty common issue when using modules.
Sometimes the program violated the One Definition Rule actually.
But sometimes it shows the compiler gives false positive diagnostics.
One often reported example is:
.. code-block:: c++
// part.cc
module;
typedef long T;
namespace ns {
inline void fun() {
(void)(T)0;
}
}
export module repro:part;
// repro.cc
module;
typedef long T;
namespace ns {
using ::T;
}
namespace ns {
inline void fun() {
(void)(T)0;
}
}
export module repro;
export import :part;
Currently the compiler complains about the inconsistent definition of `fun()` in
2 module units. This is incorrect. Since both definitions of `fun()` has the same
spelling and `T` refers to the same type entity finally. So the program should be
fine.
This is tracked in https://github.com/llvm/llvm-project/issues/78850.
Using TU-local entity in other units
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Module units are translation units. So the entities which should only be local to the
module unit itself shouldn't be used by other units in any means.
In the language side, to address the idea formally, the language specification defines
the concept of ``TU-local`` and ``exposure`` in
`basic.link/p14 <https://eel.is/c++draft/basic.link#14>`_,
`basic.link/p15 <https://eel.is/c++draft/basic.link#15>`_,
`basic.link/p16 <https://eel.is/c++draft/basic.link#16>`_,
`basic.link/p17 <https://eel.is/c++draft/basic.link#17>`_ and
`basic.link/p18 <https://eel.is/c++draft/basic.link#18>`_.
However, the compiler doesn't support these 2 ideas formally.
This results in unclear and confusing diagnostic messages.
And it is worse that the compiler may import TU-local entities to other units without any
diagnostics.
This is tracked in https://github.com/llvm/llvm-project/issues/78173.
Header Units
============
How to build projects using header unit
---------------------------------------
.. warning::
The user interfaces of header units is highly experimental. There are still
many unanswered question about how tools should interact with header units.
The user interfaces described here may change after we have progress on how
tools should support for header units.
Quick Start
~~~~~~~~~~~
For the following example,
.. code-block:: c++
import <iostream>;
int main() {
std::cout << "Hello World.\n";
}
we could compile it as
.. code-block:: console
$ clang++ -std=c++20 -xc++-system-header --precompile iostream -o iostream.pcm
$ clang++ -std=c++20 -fmodule-file=iostream.pcm main.cpp
How to produce BMIs
~~~~~~~~~~~~~~~~~~~
Similar to named modules, we could use ``--precompile`` to produce the BMI.
But we need to specify that the input file is a header by ``-xc++-system-header`` or ``-xc++-user-header``.
Also we could use `-fmodule-header={user,system}` option to produce the BMI for header units
which has suffix like `.h` or `.hh`.
The value of `-fmodule-header` means the user search path or the system search path.
The default value for `-fmodule-header` is `user`.
For example,
.. code-block:: c++
// foo.h
#include <iostream>
void Hello() {
std::cout << "Hello World.\n";
}
// use.cpp
import "foo.h";
int main() {
Hello();
}
We could compile it as:
.. code-block:: console
$ clang++ -std=c++20 -fmodule-header foo.h -o foo.pcm
$ clang++ -std=c++20 -fmodule-file=foo.pcm use.cpp
For headers which don't have a suffix, we need to pass ``-xc++-header``
(or ``-xc++-system-header`` or ``-xc++-user-header``) to mark it as a header.
For example,
.. code-block:: c++
// use.cpp
import "foo.h";
int main() {
Hello();
}
.. code-block:: console
$ clang++ -std=c++20 -fmodule-header=system -xc++-header iostream -o iostream.pcm
$ clang++ -std=c++20 -fmodule-file=iostream.pcm use.cpp
How to specify the dependent BMIs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We could use ``-fmodule-file`` to specify the BMIs, and this option may occur multiple times as well.
With the existing implementation ``-fprebuilt-module-path`` cannot be used for header units
(since they are nominally anonymous).
For header units, use ``-fmodule-file`` to include the relevant PCM file for each header unit.
This is expect to be solved in future editions of the compiler either by the tooling finding and specifying
the -fmodule-file or by the use of a module-mapper that understands how to map the header name to their PCMs.
Don't compile the BMI
~~~~~~~~~~~~~~~~~~~~~
Another difference with modules is that we can't compile the BMI from a header unit.
For example:
.. code-block:: console
$ clang++ -std=c++20 -xc++-system-header --precompile iostream -o iostream.pcm
# This is not allowed!
$ clang++ iostream.pcm -c -o iostream.o
It makes sense due to the semantics of header units, which are just like headers.
Include translation
~~~~~~~~~~~~~~~~~~~
The C++ spec allows the vendors to convert ``#include header-name`` to ``import header-name;`` when possible.
Currently, Clang would do this translation for the ``#include`` in the global module fragment.
For example, the following two examples are the same:
.. code-block:: c++
module;
import <iostream>;
export module M;
export void Hello() {
std::cout << "Hello.\n";
}
with the following one:
.. code-block:: c++
module;
#include <iostream>
export module M;
export void Hello() {
std::cout << "Hello.\n";
}
.. code-block:: console
$ clang++ -std=c++20 -xc++-system-header --precompile iostream -o iostream.pcm
$ clang++ -std=c++20 -fmodule-file=iostream.pcm --precompile M.cppm -o M.cpp
In the latter example, the Clang could find the BMI for the ``<iostream>``
so it would try to replace the ``#include <iostream>`` to ``import <iostream>;`` automatically.
Relationships between Clang modules
-----------------------------------
Header units have pretty similar semantics with Clang modules.
The semantics of both of them are like headers.
In fact, we could even "mimic" the sytle of header units by Clang modules:
.. code-block:: c++
module "iostream" {
export *
header "/path/to/libstdcxx/iostream"
}
.. code-block:: console
$ clang++ -std=c++20 -fimplicit-modules -fmodule-map-file=.modulemap main.cpp
It would be simpler if we are using libcxx:
.. code-block:: console
$ clang++ -std=c++20 main.cpp -fimplicit-modules -fimplicit-module-maps
Since there is already one
`module map <https://github.com/llvm/llvm-project/blob/main/libcxx/include/module.modulemap.in>`_
in the source of libcxx.
Then immediately leads to the question: why don't we implement header units through Clang header modules?
The main reason for this is that Clang modules have more semantics like hierarchy or
wrapping multiple headers together as a big module.
However, these things are not part of Standard C++ Header units,
and we want to avoid the impression that these additional semantics get interpreted as Standard C++ behavior.
Another reason is that there are proposals to introduce module mappers to the C++ standard
(for example, https://wg21.link/p1184r2).
If we decide to reuse Clang's modulemap, we may get in trouble once we need to introduce another module mapper.
So the final answer for why we don't reuse the interface of Clang modules for header units is that
there are some differences between header units and Clang modules and that ignoring those
differences now would likely become a problem in the future.
Discover Dependencies
=====================
Prior to modules, all the translation units can be compiled parallelly.
But it is not true for the module units. The presence of module units requires
us to compile the translation units in a (topological) order.
The clang-scan-deps scanner implemented
`P1689 paper <https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p1689r5.html>`_
to describe the order. Only named modules are supported now.
We need a compilation database to use clang-scan-deps. See
`JSON Compilation Database Format Specification <JSONCompilationDatabase.html>`_
for example. Note that the ``output`` entry is necessary for clang-scan-deps
to scan P1689 format. Here is an example:
.. code-block:: c++
//--- M.cppm
export module M;
export import :interface_part;
import :impl_part;
export int Hello();
//--- interface_part.cppm
export module M:interface_part;
export void World();
//--- Impl.cpp
module;
#include <iostream>
module M;
void Hello() {
std::cout << "Hello ";
}
//--- impl_part.cppm
module;
#include <string>
#include <iostream>
module M:impl_part;
import :interface_part;
std::string W = "World.";
void World() {
std::cout << W << std::endl;
}
//--- User.cpp
import M;
import third_party_module;
int main() {
Hello();
World();
return 0;
}
And here is the compilation database:
.. code-block:: text
[
{
"directory": ".",
"command": "<path-to-compiler-executable>/clang++ -std=c++20 M.cppm -c -o M.o",
"file": "M.cppm",
"output": "M.o"
},
{
"directory": ".",
"command": "<path-to-compiler-executable>/clang++ -std=c++20 Impl.cpp -c -o Impl.o",
"file": "Impl.cpp",
"output": "Impl.o"
},
{
"directory": ".",
"command": "<path-to-compiler-executable>/clang++ -std=c++20 impl_part.cppm -c -o impl_part.o",
"file": "impl_part.cppm",
"output": "impl_part.o"
},
{
"directory": ".",
"command": "<path-to-compiler-executable>/clang++ -std=c++20 interface_part.cppm -c -o interface_part.o",
"file": "interface_part.cppm",
"output": "interface_part.o"
},
{
"directory": ".",
"command": "<path-to-compiler-executable>/clang++ -std=c++20 User.cpp -c -o User.o",
"file": "User.cpp",
"output": "User.o"
}
]
And we can get the dependency information in P1689 format by:
.. code-block:: console
$ clang-scan-deps -format=p1689 -compilation-database P1689.json
And we will get:
.. code-block:: text
{
"revision": 0,
"rules": [
{
"primary-output": "Impl.o",
"requires": [
{
"logical-name": "M",
"source-path": "M.cppm"
}
]
},
{
"primary-output": "M.o",
"provides": [
{
"is-interface": true,
"logical-name": "M",
"source-path": "M.cppm"
}
],
"requires": [
{
"logical-name": "M:interface_part",
"source-path": "interface_part.cppm"
},
{
"logical-name": "M:impl_part",
"source-path": "impl_part.cppm"
}
]
},
{
"primary-output": "User.o",
"requires": [
{
"logical-name": "M",
"source-path": "M.cppm"
},
{
"logical-name": "third_party_module"
}
]
},
{
"primary-output": "impl_part.o",
"provides": [
{
"is-interface": false,
"logical-name": "M:impl_part",
"source-path": "impl_part.cppm"
}
],
"requires": [
{
"logical-name": "M:interface_part",
"source-path": "interface_part.cppm"
}
]
},
{
"primary-output": "interface_part.o",
"provides": [
{
"is-interface": true,
"logical-name": "M:interface_part",
"source-path": "interface_part.cppm"
}
]
}
],
"version": 1
}
See the P1689 paper for the meaning of the fields.
And if the user want a finer-grained control for any reason, e.g., to scan the generated source files,
the user can choose to get the dependency information per file. For example:
.. code-block:: console
$ clang-scan-deps -format=p1689 -- <path-to-compiler-executable>/clang++ -std=c++20 impl_part.cppm -c -o impl_part.o
And we'll get:
.. code-block:: text
{
"revision": 0,
"rules": [
{
"primary-output": "impl_part.o",
"provides": [
{
"is-interface": false,
"logical-name": "M:impl_part",
"source-path": "impl_part.cppm"
}
],
"requires": [
{
"logical-name": "M:interface_part"
}
]
}
],
"version": 1
}
In this way, we can pass the single command line options after the ``--``.
Then clang-scan-deps will extract the necessary information from the options.
Note that we need to specify the path to the compiler executable instead of saying
``clang++`` simply.
The users may want the scanner to get the transitional dependency information for headers.
Otherwise, the users have to scan twice for the project, once for headers and once for modules.
To address the requirement, clang-scan-deps will recognize the specified preprocessor options
in the given command line and generate the corresponding dependency information. For example,
.. code-block:: console
$ clang-scan-deps -format=p1689 -- ../bin/clang++ -std=c++20 impl_part.cppm -c -o impl_part.o -MD -MT impl_part.ddi -MF impl_part.dep
$ cat impl_part.dep
We will get:
.. code-block:: text
impl_part.ddi: \
/usr/include/bits/wchar.h /usr/include/bits/types/wint_t.h \
/usr/include/bits/types/mbstate_t.h \
/usr/include/bits/types/__mbstate_t.h /usr/include/bits/types/__FILE.h \
/usr/include/bits/types/FILE.h /usr/include/bits/types/locale_t.h \
/usr/include/bits/types/__locale_t.h \
...
When clang-scan-deps detects ``-MF`` option, clang-scan-deps will try to write the
dependency information for headers to the file specified by ``-MF``.
Possible Issues: Failed to find system headers
----------------------------------------------
In case the users encounter errors like ``fatal error: 'stddef.h' file not found``,
probably the specified ``<path-to-compiler-executable>/clang++`` refers to a symlink
instead a real binary. There are 4 potential solutions to the problem:
* (1) End users can resolve the issue by pointing the specified compiler executable to
the real binary instead of the symlink.
* (2) End users can invoke ``<path-to-compiler-executable>/clang++ -print-resource-dir``
to get the corresponding resource directory for your compiler and add that directory
to the include search paths manually in the build scripts.
* (3) Build systems that use a compilation database as the input for clang-scan-deps
scanner, the build system can add the flag ``--resource-dir-recipe invoke-compiler`` to
the clang-scan-deps scanner to calculate the resources directory dynamically.
The calculation happens only once for a unique ``<path-to-compiler-executable>/clang++``.
* (4) For build systems that invokes the clang-scan-deps scanner per file, repeatedly
calculating the resource directory may be inefficient. In such cases, the build
system can cache the resource directory by itself and pass ``-resource-dir <resource-dir>``
explicitly in the command line options:
.. code-block:: console
$ clang-scan-deps -format=p1689 -- <path-to-compiler-executable>/clang++ -std=c++20 -resource-dir <resource-dir> mod.cppm -c -o mod.o
Import modules with clang-repl
==============================
We're able to import C++20 named modules with clang-repl.
Let's start with a simple example:
.. code-block:: c++
// M.cppm
export module M;
export const char* Hello() {
return "Hello Interpreter for Modules!";
}
We still need to compile the named module in ahead.
.. code-block:: console
$ clang++ -std=c++20 M.cppm --precompile -o M.pcm
$ clang++ M.pcm -c -o M.o
$ clang++ -shared M.o -o libM.so
Note that we need to compile the module unit into a dynamic library so that the clang-repl
can load the object files of the module units.
Then we are able to import module ``M`` in clang-repl.
.. code-block:: console
$ clang-repl -Xcc=-std=c++20 -Xcc=-fprebuilt-module-path=.
# We need to load the dynamic library first before importing the modules.
clang-repl> %lib libM.so
clang-repl> import M;
clang-repl> extern "C" int printf(const char *, ...);
clang-repl> printf("%s\n", Hello());
Hello Interpreter for Modules!
clang-repl> %quit
Possible Questions
==================
How modules speed up compilation
--------------------------------
A classic theory for the reason why modules speed up the compilation is:
if there are ``n`` headers and ``m`` source files and each header is included by each source file,
then the complexity of the compilation is ``O(n*m)``;
But if there are ``n`` module interfaces and ``m`` source files, the complexity of the compilation is
``O(n+m)``. So, using modules would be a big win when scaling.
In a simpler word, we could get rid of many redundant compilations by using modules.
Roughly, this theory is correct. But the problem is that it is too rough.
The behavior depends on the optimization level, as we will illustrate below.
First is ``O0``. The compilation process is described in the following graph.
.. code-block:: none
├-------------frontend----------┼-------------middle end----------------┼----backend----┤
│ │ │ │
└---parsing----sema----codegen--┴----- transformations ---- codegen ----┴---- codegen --┘
┌---------------------------------------------------------------------------------------┐
| │
| source file │
| │
└---------------------------------------------------------------------------------------┘
┌--------┐
│ │
│imported│
│ │
│ code │
│ │
└--------┘
Here we can see that the source file (could be a non-module unit or a module unit) would get processed by the
whole pipeline.
But the imported code would only get involved in semantic analysis, which is mainly about name lookup,
overload resolution and template instantiation.
All of these processes are fast relative to the whole compilation process.
More importantly, the imported code only needs to be processed once in frontend code generation,
as well as the whole middle end and backend.
So we could get a big win for the compilation time in O0.
But with optimizations, things are different:
(we omit ``code generation`` part for each end due to the limited space)
.. code-block:: none
├-------- frontend ---------┼--------------- middle end --------------------┼------ backend ----┤
│ │ │ │
└--- parsing ---- sema -----┴--- optimizations --- IPO ---- optimizations---┴--- optimizations -┘
┌-----------------------------------------------------------------------------------------------┐
│ │
│ source file │
│ │
└-----------------------------------------------------------------------------------------------┘
┌---------------------------------------┐
│ │
│ │
│ imported code │
│ │
│ │
└---------------------------------------┘
It would be very unfortunate if we end up with worse performance after using modules.
The main concern is that when we compile a source file, the compiler needs to see the function body
of imported module units so that it can perform IPO (InterProcedural Optimization, primarily inlining
in practice) to optimize functions in current source file with the help of the information provided by
the imported module units.
In other words, the imported code would be processed again and again in importee units
by optimizations (including IPO itself).
The optimizations before IPO and the IPO itself are the most time-consuming part in whole compilation process.
So from this perspective, we might not be able to get the improvements described in the theory.
But we could still save the time for optimizations after IPO and the whole backend.
Overall, at ``O0`` the implementations of functions defined in a module will not impact module users,
but at higher optimization levels the definitions of such functions are provided to user compilations for the
purposes of optimization (but definitions of these functions are still not included in the use's object file)-
this means the build speedup at higher optimization levels may be lower than expected given ``O0`` experience,
but does provide by more optimization opportunities.
Interoperability with Clang Modules
-----------------------------------
We **wish** to support clang modules and standard c++ modules at the same time,
but the mixed using form is not well used/tested yet.
Please file new github issues as you find interoperability problems.
|