176 Commits

Autor SHA1 Mensagem Data
Ken Raffenetti ec4cc89f5e build: Move benchmark generation to autogen.sh in testsuite
These are only needed by the testsuite. If mydef is unavailable for some
reason, e.g. running autogen.sh from a testsuite-only tarball,
regeneration of the benchmarks will be skipped.
2024-10-16 11:01:24 -05:00
Hui Zhou 6633f0a001 autogen: convert mydef code in autogen
We could add rules to directly work with mydef code in Makefile, but
convert the code in autogen removes the mydef dependency.

Also fix a spelling error.
2024-10-01 22:42:55 -05:00
Rob Latham f4ee998f9a build: more informative failure message
If autoconf detection fails, see if we can provide a bit more of a hint
than just "bzzt wrong"
2024-06-25 10:41:57 -05:00
wkliao a68350b483 more on fixing detection of python version 3 2024-05-01 07:53:35 -07:00
Hui Zhou 20b2a7f557 binding/c: add back -single-source option
The single source option generates huge c_binding.c. It may be desirable
to opt for multiple source files instead.

TODO: I still think we should do -single-source by default, but it is nice
to have the option.

There are functions removed or deprecated with replacement functions,
which we implemented by calling MPI_xxx_internal functions. Thus
we need generate all replaced functions together to avoid making those
static internal functions external.
2024-03-07 12:42:58 -06:00
Hui Zhou c7a27e7994 build: autogen f08 autoconf file
The autoconf .in files have to generated inside the source tree. Thus we
cannot generate them during configure.

Also, use AC_CONFIG_COMMANDS_PRE to run python scripts inside configure,
thus will abort configure upon errors.
2023-12-18 16:00:38 -06:00
Hui Zhou cafa7be9f1 ABI: add gen_abi.py
Then internal code need deal with both the types from ABI header and the
actual types used by MPICH. Generate mpi_abi_internal.h from mpi_abi.h
by renaming all MPI_ typenames into ABI_ prefix. Generate mpi_abi_util.c
to initialize an internal table for builtin datatypes and ops
conversions.

Generates:
    src/binding/abi/mpi_abi_internal.h
    src/binding/abi/mpi_abi_util.c
2023-12-12 22:42:43 -06:00
Hui Zhou d1b3f5da2d f08: replace buildiface in gen_binding_f08.py
Similar to previous commit, we replace the perl script that generates
src/binding/fortran/use_mpi_f08/mpi_f08_compile_constants.f90.in in
gen_binding_f08.py.

This also added the missing MPI 4.0/4.1 symbols including:
    MPI_COMBINER_VALUE_INDEX
    MPI_COMM_TYPE_HW_GUIDED
    MPI_COMM_TYPE_HW_UNGUIDED
    MPI_COMM_TYPE_RESOURCE_GUIDED
    MPI_ERRORS_ABORT
    MPI_SESSION_NULL
    MPI_MAX_PSETNAME_LEN
    MPI_MAX_STRINGTAG_LEN

In addition, previous perl script failed to account for the "MAX"
constants need to be 1-less than the C correspondent to account for the
terminating NUL in C string.
2023-12-07 22:55:13 -06:00
Hui Zhou 2b8b50ca95 f77: replace buildiface in python scripts
The perl script src/binding/fortran/mpif_h/buildiface generates
mpif.h.in from scraping mpi.h.in and mpio.h.in. The old script has
become difficult to maintain and it is missing a few newly added symbols
including -
    MPI_COMBINER_VALUE_INDEX
    MPI_COMM_TYPE_HW_GUIDED
    MPI_COMM_TYPE_HW_UNGUIDED
    MPI_COMM_TYPE_RESOURCE_GUIDED
    MPI_ERRORS_ABORT
    MPI_MAX_PSET_NAME_LEN
    MPI_MAX_STRINGTAG_LEN
    MPI_SESSION_NULL

In addition, it needs declare all functions that does not return
INTEGER. It was missing MPI_Aint_add and MPI_Aint_diff.
2023-12-07 10:50:57 -06:00
Hui Zhou 986e9541f3 configure: only substitute fortran files if configured
If user add --disable-fortran option, make configure not depend on the
fortran files that maybe missing from the autogen.
2023-07-26 11:18:04 -05:00
Hui Zhou ccc919430e binding/c: output a single c_binding.c
Rather than generate one source code for each MPI function, generate all
the binding layer in a single c_binding.c. It compiles faster and allows
future refactoring inside the binding layer.
2023-05-01 15:08:10 -05:00
Hui Zhou 3c13018bc5 autogen: sync src/pm/hydra/modules/pmi/mpl/confdb
Unfortunately for full autogen, we need sync every confdb.
2022-11-15 12:13:03 -06:00
Hui Zhou 164c87af9c build: allow user to set PYTHON
Allow user to manually set environment variable PYTHON to the Python 3
executable in case we are not able to find it.
2022-11-14 10:31:44 -06:00
Hui Zhou c0c834dc19 hydra: import pmi into hydra
Hydra shares the wire protocol with the pmi client. Importing pmi avoids
duplicating code and adds consistency.
2022-08-23 11:32:10 -05:00
Hui Zhou a40c62ee95 pm: remove hydra2
At this point it is clear that we are going to keep developing hydra
rather than fixing the missing features in hydra2.
2022-07-20 08:47:44 -05:00
Yanfei Guo 4b6afc4f70 autogen: update autoconf version requirement to match configure.ac 2022-07-19 16:03:39 -05:00
Hui Zhou f588fc4ce6 hydra: fix hwloc paths and move them to modules dir
It is unnecessary to put embedded library to deep nested paths. Put them
all in modules dir, similar to how mpich does it.

The hwloc path was previously broken when we rearranged the source tree.
It didn't show in our testing because by default, we reuse the hwloc
compiled by main mpich.
2022-07-13 14:18:19 -05:00
Hui Zhou 06ece5683f maint: hard code maint/cvardirs.in in extractcvars
Practically one would never need to run extractcvars with different
source directories. Hard code the directory list in the extractcvars
script, simplify a few unnecessary steps.
2022-06-17 17:23:13 -05:00
Hui Zhou 1b75b499e1 misc: remove izem
We no longer use izem since workq has been removed.
2022-06-07 13:45:32 -05:00
Ken Raffenetti fd14090d50 autogen: Silence output during python3 check
Send any error output to /dev/null and echo a dummy value to avoid
"unary operator expected" warnings.
2022-04-26 12:55:04 -05:00
Hui Zhou 3e34b6789d autogen: add option to autogen yaksa with less depth
Yaksa will take a long time to compile at the default
`-pup-max-nesting=3`. Add an option to generate shallower depth for
faster build.
2022-04-19 13:40:26 -05:00
Hui Zhou 4e61b73173 autogen: add message when submodule is missing
Add message to advise users to run git submodule checkout when they just
cloned the repository.
2022-03-28 16:53:48 -05:00
Hui Zhou 1836650000 pmi: make pmi a submodule
Make pmi a submodule like mpl and romio. This allows to just build and install
libpmi, allowing applications and tests to use PMI interface without
mpi.
2022-03-28 16:29:58 -05:00
Hui Zhou 73d6a89e74 autogen: use --with-doc to generate man sources
Typically we only need generate man sources when preparing release
tarball.
2022-02-16 16:34:38 -06:00
Hui Zhou 02cd7a14b2 maint: fix extracterrmsgs
We have been manually setting MPI error class index in
src/mpi/errhan/baseerrnames.txt, and it has been out-of-sync with the
values defined in mpi.h. This commit does following:

* Remove the index from baseerrnames.txt and directly load the defined
  value from mpi.h instead.

* Add UNKNOWN entry to map the missing entries in baseerrnames.txt.

* When autogen.sh fails to extract error messages, do not generate dummy
  defmsg.h, fail instead. The dummy header will break the error class
  message translation anyway.
2022-02-07 15:53:10 -06:00
Hui Zhou 66f8df011b configure: generate f90 bindings in configure
This allows checking compiler features, for example, the directives to
ignore TKR check.
2022-02-04 17:32:45 -06:00
Hui Zhou 94ccecd7bc autogen: add running maint/gen_binding_f90.py
We'll move into configure later to accept config options.

We also remove the replacement of return types for MPI_Wtime/MPI_Wtick
for now. They'll be taken care of once we move the codegen into
configure.
2022-02-04 17:32:45 -06:00
Hui Zhou ccd6f036a1 autogen: skip autotools version check in quick mode
For CI jobs where we always test with a pre-configured environment,
checking autotools versions are unnecessary. Since the checking is very
slow, it makes sense to skip them with `-quick` option.
2022-02-04 12:25:17 -06:00
Hui Zhou 3ebb447e8a autogen: in quick mode generate single c binding source
This will accelerate the compilation.
2022-02-04 12:25:17 -06:00
Ken Raffenetti 96fa0bd308 maint: Add libtool.m4 patch for NVIDIA HPC Compilers
Patch adapted from the libtool patches
list. https://lists.gnu.org/archive/html/libtool-patches/2020-08/msg00000.html.
2022-01-18 13:27:15 -06:00
Hui Zhou f834314286 autogen: skip hwloc and json-c in quick mode 2022-01-17 11:21:39 -06:00
Hui Zhou caff745425 autogen: untar modules.tar.gz in quick mode 2022-01-17 11:21:39 -06:00
Hui Zhou c8e80ad5d5 autogen: minor cleanup 2021-12-22 10:28:39 -06:00
Hui Zhou 4ef8c92c1e autogen: refactor fn_copy_confdb_etc 2021-12-22 10:28:39 -06:00
Hui Zhou eca7a33ede autogen: refactor set_externals
Check and set externals array is a preprequisite function
2021-12-22 10:28:39 -06:00
Hui Zhou 2f189f9cb5 autogen: refactor fn_build_configure
Having individula fn_autoreconf_amdir and fn_autogen_external does not
fit the -do=xxx semantics. Refactor and use fn_build_configure instead.
This brings the original -do=build_configure option back.
2021-12-22 10:28:39 -06:00
Hui Zhou b7ba89c617 autogen: refactor gen_subcfg romio_glue and json_gen 2021-12-22 10:28:39 -06:00
Hui Zhou e2a1021437 autogen: refactor python steps
It can be useful to run inidividual api generation steps. Add the PYTHON
check into each function so the dependency is managed automatically.
2021-12-22 10:28:39 -06:00
Hui Zhou 078282b06b autogen: make set_autotools a prerequisite function
A prerequisite function is always checked to allow individual step
function to be run indenpendently.
2021-12-22 10:28:39 -06:00
Hui Zhou f65264dc94 autogen: consolidate --with-xxx options 2021-12-22 10:28:39 -06:00
Hui Zhou 3075f6f8a4 autogen: let -do=xxx execute a single step function
Mixing the --with-xxx and -do=xxx options makes the script logic very
complex since there are extensible steps and many implicit dependency
conditions. Instead, let -do=xxx just execute a single step, which I
believe it is the original intention of this option. We assume that when
this option is used, the developer knows the dependency are already done
or checked.
2021-12-22 10:28:39 -06:00
Hui Zhou 894742438c autogen: indentation fix 2021-12-22 10:28:39 -06:00
Hui Zhou 51a4fd9b14 autogen: move global options to the front
The global option variables are used in various step functions. Move
them to the front of the script helps understanding the structure.
2021-12-22 10:28:39 -06:00
Hui Zhou 01877e3c8e autogen: remove single step options and checks
The current autogen does not make the steps clear, and it is confusing
to run a single step. Remove the option and checks for now.

TODO: refactor the single steps into individual functions. When -do=xxx
is given, run that function and exit.
2021-12-22 10:28:39 -06:00
Hui Zhou 7d21482abf autogen: refactor functions to check prerequisite 2021-12-22 10:28:39 -06:00
Hui Zhou 14f7c05ec6 autogen: refactor fn_check_autotools 2021-12-22 10:28:39 -06:00
Hui Zhou 48c2c8c966 autogen: refactor fn_maint_version and fn_update_README 2021-12-22 10:28:39 -06:00
Hui Zhou e46ec68cc7 autogen: refactor binding generations 2021-12-22 10:28:39 -06:00
Hui Zhou 04eaa356c5 autogen: refactor fn_autoreconf_amdir 2021-12-22 10:28:39 -06:00
Hui Zhou 701e3417bc autogen: refactor fn_getcvars
We'll run configure in maint if it isn't run already.
2021-12-22 10:28:39 -06:00