autoconf: New test for openpty et al.

We may need to #include <libutil.h>, and/or link with -lutil, to use
openpty, login_tty, and the like.  Provide INCLUDE_LIBUTIL_H
(preprocessor constant, not always defined) and PTYFUNCS_LIBS
(makefile variable).

We link libxl against PTYFUNCS_LIBS (which comes from autoconf) rather
than UTIL_LIBS, and #include <libutil.h> where appropriate.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>

Changes since v7:
 * Actually include the call to AX_CHECK_PTYFUNCS in this patch,
   not the previous one, and regenerate configure accordingly.

Changes since v6:
 * Put failure macro call in correct place so it might actually happen.
 * Try both with -lutil and without.
 * Patch now contains update for config.h.in.
Committed-by: Ian Jackson <Ian.Jackson@eu.citrix.com>

1 files changed, 28 insertions, 0 deletions

diff --git a/tools/m4/ptyfuncs.m4 b/tools/m4/ptyfuncs.m4
new file mode 100644
index 0000000000..7581704f3e
--- /dev/null
+++ b/tools/m4/ptyfuncs.m4
@@ -0,0 +1,28 @@
+AC_DEFUN([AX_CHECK_PTYFUNCS], [
+    AC_CHECK_HEADER([libutil.h],[
+      AC_DEFINE([INCLUDE_LIBUTIL_H],[<libutil.h>],[libutil header file name])
+    ])
+    AC_CACHE_CHECK([for openpty et al], [ax_cv_ptyfuncs_libs], [
+        for ax_cv_ptyfuncs_libs in -lutil "" NOT_FOUND; do
+            if test "x$ax_cv_ptyfuncs_libs" = "xNOT_FOUND"; then
+                AC_MSG_FAILURE([Unable to find library for openpty and login_tty])
+            fi
+            AX_SAVEVAR_SAVE(LIBS)
+            LIBS="$LIBS $ax_cv_ptyfuncs_libs"
+            AC_LINK_IFELSE([
+#ifdef INCLUDE_LIBUTIL_H
+#include INCLUDE_LIBUTIL_H
+#endif
+int main(void) {
+  openpty(0,0,0,0,0);
+  login_tty(0);
+}
+],[
+                break
+            ],[])
+            AX_SAVEVAR_RESTORE(LIBS)
+        done
+    ])
+    PTYFUNCS_LIBS="$ax_cv_ptyfuncs_libs"
+    AC_SUBST(PTYFUNCS_LIBS)
+])