aboutsummaryrefslogtreecommitdiffstats
path: root/3rdparty/pybind11/docs/upgrade.rst
blob: 87bcebee2c410f9b4ae623a71a9bd10eedf65137 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
Upgrade guide
#############

This is a companion guide to the :doc:`changelog`. While the changelog briefly
lists all of the new features, improvements and bug fixes, this upgrade guide
focuses only the subset which directly impacts your experience when upgrading
to a new version. But it goes into more detail. This includes things like
deprecated APIs and their replacements, build system changes, general code
modernization and other useful information.

.. _upgrade-guide-2.6:

v2.6
====

Usage of the ``PYBIND11_OVERLOAD*`` macros and ``get_overload`` function should
be replaced by ``PYBIND11_OVERRIDE*`` and ``get_override``. In the future, the
old macros may be deprecated and removed.

``py::module`` has been renamed ``py::module_``, but a backward compatible
typedef has been included. This change was to avoid a language change in C++20
that requires unqualified ``module`` not be placed at the start of a logical
line. Qualified usage is unaffected and the typedef will remain unless the
C++ language rules change again.

The public constructors of ``py::module_`` have been deprecated. Use
``PYBIND11_MODULE`` or ``module_::create_extension_module`` instead.

An error is now thrown when ``__init__`` is forgotten on subclasses. This was
incorrect before, but was not checked. Add a call to ``__init__`` if it is
missing.

A ``py::type_error`` is now thrown when casting to a subclass (like
``py::bytes`` from ``py::object``) if the conversion is not valid. Make a valid
conversion instead.

The undocumented ``h.get_type()`` method has been deprecated and replaced by
``py::type::of(h)``.

Enums now have a ``__str__`` method pre-defined; if you want to override it,
the simplest fix is to add the new ``py::prepend()`` tag when defining
``"__str__"``.

If ``__eq__`` defined but not ``__hash__``, ``__hash__`` is now set to
``None``, as in normal CPython. You should add ``__hash__`` if you intended the
class to be hashable, possibly using the new ``py::hash`` shortcut.

The constructors for ``py::array`` now always take signed integers for size,
for consistency. This may lead to compiler warnings on some systems. Cast to
``py::ssize_t`` instead of ``std::size_t``.

The ``tools/clang`` submodule and ``tools/mkdoc.py`` have been moved to a
standalone package, `pybind11-mkdoc`_. If you were using those tools, please
use them via a pip install from the new location.

The ``pybind11`` package on PyPI no longer fills the wheel "headers" slot - if
you were using the headers from this slot, they are available by requesting the
``global`` extra, that is, ``pip install "pybind11[global]"``. (Most users will
be unaffected, as the ``pybind11/include`` location is reported by ``python -m
pybind11 --includes`` and ``pybind11.get_include()`` is still correct and has
not changed since 2.5).

.. _pybind11-mkdoc: https://github.com/pybind/pybind11-mkdoc

CMake support:
--------------

The minimum required version of CMake is now 3.4.  Several details of the CMake
support have been deprecated; warnings will be shown if you need to change
something. The changes are:

* ``PYBIND11_CPP_STANDARD=<platform-flag>`` is deprecated, please use
  ``CMAKE_CXX_STANDARD=<number>`` instead, or any other valid CMake CXX or CUDA
  standard selection method, like ``target_compile_features``.

* If you do not request a standard, pybind11 targets will compile with the
  compiler default, but not less than C++11, instead of forcing C++14 always.
  If you depend on the old behavior, please use ``set(CMAKE_CXX_STANDARD 14 CACHE STRING "")``
  instead.

* Direct ``pybind11::module`` usage should always be accompanied by at least
  ``set(CMAKE_CXX_VISIBILITY_PRESET hidden)`` or similar - it used to try to
  manually force this compiler flag (but not correctly on all compilers or with
  CUDA).

* ``pybind11_add_module``'s ``SYSTEM`` argument is deprecated and does nothing;
  linking now behaves like other imported libraries consistently in both
  config and submodule mode, and behaves like a ``SYSTEM`` library by
  default.

* If ``PYTHON_EXECUTABLE`` is not set, virtual environments (``venv``,
  ``virtualenv``, and ``conda``) are prioritized over the standard search
  (similar to the new FindPython mode).

In addition, the following changes may be of interest:

* ``CMAKE_INTERPROCEDURAL_OPTIMIZATION`` will be respected by
  ``pybind11_add_module`` if set instead of linking to ``pybind11::lto`` or
  ``pybind11::thin_lto``.

* Using ``find_package(Python COMPONENTS Interpreter Development)`` before
  pybind11 will cause pybind11 to use the new Python mechanisms instead of its
  own custom search, based on a patched version of classic ``FindPythonInterp``
  / ``FindPythonLibs``. In the future, this may become the default. A recent
  (3.15+ or 3.18.2+) version of CMake is recommended.



v2.5
====

The Python package now includes the headers as data in the package itself, as
well as in the "headers" wheel slot. ``pybind11 --includes`` and
``pybind11.get_include()`` report the new location, which is always correct
regardless of how pybind11 was installed, making the old ``user=`` argument
meaningless. If you are not using the function to get the location already, you
are encouraged to switch to the package location.


v2.2
====

Deprecation of the ``PYBIND11_PLUGIN`` macro
--------------------------------------------

``PYBIND11_MODULE`` is now the preferred way to create module entry points.
The old macro emits a compile-time deprecation warning.

.. code-block:: cpp

    // old
    PYBIND11_PLUGIN(example) {
        py::module m("example", "documentation string");

        m.def("add", [](int a, int b) { return a + b; });

        return m.ptr();
    }

    // new
    PYBIND11_MODULE(example, m) {
        m.doc() = "documentation string"; // optional

        m.def("add", [](int a, int b) { return a + b; });
    }


New API for defining custom constructors and pickling functions
---------------------------------------------------------------

The old placement-new custom constructors have been deprecated. The new approach
uses ``py::init()`` and factory functions to greatly improve type safety.

Placement-new can be called accidentally with an incompatible type (without any
compiler errors or warnings), or it can initialize the same object multiple times
if not careful with the Python-side ``__init__`` calls. The new-style custom
constructors prevent such mistakes. See :ref:`custom_constructors` for details.

.. code-block:: cpp

    // old -- deprecated (runtime warning shown only in debug mode)
    py::class<Foo>(m, "Foo")
        .def("__init__", [](Foo &self, ...) {
            new (&self) Foo(...); // uses placement-new
        });

    // new
    py::class<Foo>(m, "Foo")
        .def(py::init([](...) { // Note: no `self` argument
            return new Foo(...); // return by raw pointer
            // or: return std::make_unique<Foo>(...); // return by holder
            // or: return Foo(...); // return by value (move constructor)
        }));

Mirroring the custom constructor changes, ``py::pickle()`` is now the preferred
way to get and set object state. See :ref:`pickling` for details.

.. code-block:: cpp

    // old -- deprecated (runtime warning shown only in debug mode)
    py::class<Foo>(m, "Foo")
        ...
        .def("__getstate__", [](const Foo &self) {
            return py::make_tuple(self.value1(), self.value2(), ...);
        })
        .def("__setstate__", [](Foo &self, py::tuple t) {
            new (&self) Foo(t[0].cast<std::string>(), ...);
        });

    // new
    py::class<Foo>(m, "Foo")
        ...
        .def(py::pickle(
            [](const Foo &self) { // __getstate__
                return py::make_tuple(f.value1(), f.value2(), ...); // unchanged
            },
            [](py::tuple t) { // __setstate__, note: no `self` argument
                return new Foo(t[0].cast<std::string>(), ...);
                // or: return std::make_unique<Foo>(...); // return by holder
                // or: return Foo(...); // return by value (move constructor)
            }
        ));

For both the constructors and pickling, warnings are shown at module
initialization time (on import, not when the functions are called).
They're only visible when compiled in debug mode. Sample warning:

.. code-block:: none

    pybind11-bound class 'mymodule.Foo' is using an old-style placement-new '__init__'
    which has been deprecated. See the upgrade guide in pybind11's docs.


Stricter enforcement of hidden symbol visibility for pybind11 modules
---------------------------------------------------------------------

pybind11 now tries to actively enforce hidden symbol visibility for modules.
If you're using either one of pybind11's :doc:`CMake or Python build systems
<compiling>` (the two example repositories) and you haven't been exporting any
symbols, there's nothing to be concerned about. All the changes have been done
transparently in the background. If you were building manually or relied on
specific default visibility, read on.

Setting default symbol visibility to *hidden* has always been recommended for
pybind11 (see :ref:`faq:symhidden`). On Linux and macOS, hidden symbol
visibility (in conjunction with the ``strip`` utility) yields much smaller
module binaries. `CPython's extension docs`_ also recommend hiding symbols
by default, with the goal of avoiding symbol name clashes between modules.
Starting with v2.2, pybind11 enforces this more strictly: (1) by declaring
all symbols inside the ``pybind11`` namespace as hidden and (2) by including
the ``-fvisibility=hidden`` flag on Linux and macOS (only for extension
modules, not for embedding the interpreter).

.. _CPython's extension docs: https://docs.python.org/3/extending/extending.html#providing-a-c-api-for-an-extension-module

The namespace-scope hidden visibility is done automatically in pybind11's
headers and it's generally transparent to users. It ensures that:

* Modules compiled with different pybind11 versions don't clash with each other.

* Some new features, like ``py::module_local`` bindings, can work as intended.

The ``-fvisibility=hidden`` flag applies the same visibility to user bindings
outside of the ``pybind11`` namespace. It's now set automatic by pybind11's
CMake and Python build systems, but this needs to be done manually by users
of other build systems. Adding this flag:

* Minimizes the chances of symbol conflicts between modules. E.g. if two
  unrelated modules were statically linked to different (ABI-incompatible)
  versions of the same third-party library, a symbol clash would be likely
  (and would end with unpredictable results).

* Produces smaller binaries on Linux and macOS, as pointed out previously.

Within pybind11's CMake build system, ``pybind11_add_module`` has always been
setting the ``-fvisibility=hidden`` flag in release mode. From now on, it's
being applied unconditionally, even in debug mode and it can no longer be opted
out of with the ``NO_EXTRAS`` option. The ``pybind11::module`` target now also
adds this flag to it's interface. The ``pybind11::embed`` target is unchanged.

The most significant change here is for the ``pybind11::module`` target. If you
were previously relying on default visibility, i.e. if your Python module was
doubling as a shared library with dependents, you'll need to either export
symbols manually (recommended for cross-platform libraries) or factor out the
shared library (and have the Python module link to it like the other
dependents). As a temporary workaround, you can also restore default visibility
using the CMake code below, but this is not recommended in the long run:

.. code-block:: cmake

    target_link_libraries(mymodule PRIVATE pybind11::module)

    add_library(restore_default_visibility INTERFACE)
    target_compile_options(restore_default_visibility INTERFACE -fvisibility=default)
    target_link_libraries(mymodule PRIVATE restore_default_visibility)


Local STL container bindings
----------------------------

Previous pybind11 versions could only bind types globally -- all pybind11
modules, even unrelated ones, would have access to the same exported types.
However, this would also result in a conflict if two modules exported the
same C++ type, which is especially problematic for very common types, e.g.
``std::vector<int>``. :ref:`module_local` were added to resolve this (see
that section for a complete usage guide).

``py::class_`` still defaults to global bindings (because these types are
usually unique across modules), however in order to avoid clashes of opaque
types, ``py::bind_vector`` and ``py::bind_map`` will now bind STL containers
as ``py::module_local`` if their elements are: builtins (``int``, ``float``,
etc.), not bound using ``py::class_``, or bound as ``py::module_local``. For
example, this change allows multiple modules to bind ``std::vector<int>``
without causing conflicts. See :ref:`stl_bind` for more details.

When upgrading to this version, if you have multiple modules which depend on
a single global binding of an STL container, note that all modules can still
accept foreign  ``py::module_local`` types in the direction of Python-to-C++.
The locality only affects the C++-to-Python direction. If this is needed in
multiple modules, you'll need to either:

* Add a copy of the same STL binding to all of the modules which need it.

* Restore the global status of that single binding by marking it
  ``py::module_local(false)``.

The latter is an easy workaround, but in the long run it would be best to
localize all common type bindings in order to avoid conflicts with
third-party modules.


Negative strides for Python buffer objects and numpy arrays
-----------------------------------------------------------

Support for negative strides required changing the integer type from unsigned
to signed in the interfaces of ``py::buffer_info`` and ``py::array``. If you
have compiler warnings enabled, you may notice some new conversion warnings
after upgrading. These can be resolved using ``static_cast``.


Deprecation of some ``py::object`` APIs
---------------------------------------

To compare ``py::object`` instances by pointer, you should now use
``obj1.is(obj2)`` which is equivalent to ``obj1 is obj2`` in Python.
Previously, pybind11 used ``operator==`` for this (``obj1 == obj2``), but
that could be confusing and is now deprecated (so that it can eventually
be replaced with proper rich object comparison in a future release).

For classes which inherit from ``py::object``, ``borrowed`` and ``stolen``
were previously available as protected constructor tags. Now the types
should be used directly instead: ``borrowed_t{}`` and ``stolen_t{}``
(`#771 <https://github.com/pybind/pybind11/pull/771>`_).


Stricter compile-time error checking
------------------------------------

Some error checks have been moved from run time to compile time. Notably,
automatic conversion of ``std::shared_ptr<T>`` is not possible when ``T`` is
not directly registered with ``py::class_<T>`` (e.g. ``std::shared_ptr<int>``
or ``std::shared_ptr<std::vector<T>>`` are not automatically convertible).
Attempting to bind a function with such arguments now results in a compile-time
error instead of waiting to fail at run time.

``py::init<...>()`` constructor definitions are also stricter and now prevent
bindings which could cause unexpected behavior:

.. code-block:: cpp

    struct Example {
        Example(int &);
    };

    py::class_<Example>(m, "Example")
        .def(py::init<int &>()); // OK, exact match
        // .def(py::init<int>()); // compile-time error, mismatch

A non-``const`` lvalue reference is not allowed to bind to an rvalue. However,
note that a constructor taking ``const T &`` can still be registered using
``py::init<T>()`` because a ``const`` lvalue reference can bind to an rvalue.

v2.1
====

Minimum compiler versions are enforced at compile time
------------------------------------------------------

The minimums also apply to v2.0 but the check is now explicit and a compile-time
error is raised if the compiler does not meet the requirements:

* GCC >= 4.8
* clang >= 3.3 (appleclang >= 5.0)
* MSVC >= 2015u3
* Intel C++ >= 15.0


The ``py::metaclass`` attribute is not required for static properties
---------------------------------------------------------------------

Binding classes with static properties is now possible by default. The
zero-parameter version of ``py::metaclass()`` is deprecated. However, a new
one-parameter ``py::metaclass(python_type)`` version was added for rare
cases when a custom metaclass is needed to override pybind11's default.

.. code-block:: cpp

    // old -- emits a deprecation warning
    py::class_<Foo>(m, "Foo", py::metaclass())
        .def_property_readonly_static("foo", ...);

    // new -- static properties work without the attribute
    py::class_<Foo>(m, "Foo")
        .def_property_readonly_static("foo", ...);

    // new -- advanced feature, override pybind11's default metaclass
    py::class_<Bar>(m, "Bar", py::metaclass(custom_python_type))
        ...


v2.0
====

Breaking changes in ``py::class_``
----------------------------------

These changes were necessary to make type definitions in pybind11
future-proof, to support PyPy via its ``cpyext`` mechanism (`#527
<https://github.com/pybind/pybind11/pull/527>`_), and to improve efficiency
(`rev. 86d825 <https://github.com/pybind/pybind11/commit/86d825>`_).

1. Declarations of types that provide access via the buffer protocol must
   now include the ``py::buffer_protocol()`` annotation as an argument to
   the ``py::class_`` constructor.

   .. code-block:: cpp

       py::class_<Matrix>("Matrix", py::buffer_protocol())
           .def(py::init<...>())
           .def_buffer(...);

2. Classes which include static properties (e.g. ``def_readwrite_static()``)
   must now include the ``py::metaclass()`` attribute. Note: this requirement
   has since been removed in v2.1. If you're upgrading from 1.x, it's
   recommended to skip directly to v2.1 or newer.

3. This version of pybind11 uses a redesigned mechanism for instantiating
   trampoline classes that are used to override virtual methods from within
   Python. This led to the following user-visible syntax change:

   .. code-block:: cpp

       // old v1.x syntax
       py::class_<TrampolineClass>("MyClass")
           .alias<MyClass>()
           ...

       // new v2.x syntax
       py::class_<MyClass, TrampolineClass>("MyClass")
           ...

   Importantly, both the original and the trampoline class are now specified
   as arguments to the ``py::class_`` template, and the ``alias<..>()`` call
   is gone. The new scheme has zero overhead in cases when Python doesn't
   override any functions of the underlying C++ class.
   `rev. 86d825 <https://github.com/pybind/pybind11/commit/86d825>`_.

   The class type must be the first template argument given to ``py::class_``
   while the trampoline can be mixed in arbitrary order with other arguments
   (see the following section).


Deprecation of the ``py::base<T>()`` attribute
----------------------------------------------

``py::base<T>()`` was deprecated in favor of specifying ``T`` as a template
argument to ``py::class_``. This new syntax also supports multiple inheritance.
Note that, while the type being exported must be the first argument in the
``py::class_<Class, ...>`` template, the order of the following types (bases,
holder and/or trampoline) is not important.

.. code-block:: cpp

    // old v1.x
    py::class_<Derived>("Derived", py::base<Base>());

    // new v2.x
    py::class_<Derived, Base>("Derived");

    // new -- multiple inheritance
    py::class_<Derived, Base1, Base2>("Derived");

    // new -- apart from `Derived` the argument order can be arbitrary
    py::class_<Derived, Base1, Holder, Base2, Trampoline>("Derived");


Out-of-the-box support for ``std::shared_ptr``
----------------------------------------------

The relevant type caster is now built in, so it's no longer necessary to
include a declaration of the form:

.. code-block:: cpp

    PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>)

Continuing to do so won’t cause an error or even a deprecation warning,
but it's completely redundant.


Deprecation of a few ``py::object`` APIs
----------------------------------------

All of the old-style calls emit deprecation warnings.

+---------------------------------------+---------------------------------------------+
|  Old syntax                           |  New syntax                                 |
+=======================================+=============================================+
| ``obj.call(args...)``                 | ``obj(args...)``                            |
+---------------------------------------+---------------------------------------------+
| ``obj.str()``                         | ``py::str(obj)``                            |
+---------------------------------------+---------------------------------------------+
| ``auto l = py::list(obj); l.check()`` | ``py::isinstance<py::list>(obj)``           |
+---------------------------------------+---------------------------------------------+
| ``py::object(ptr, true)``             | ``py::reinterpret_borrow<py::object>(ptr)`` |
+---------------------------------------+---------------------------------------------+
| ``py::object(ptr, false)``            | ``py::reinterpret_steal<py::object>(ptr)``  |
+---------------------------------------+---------------------------------------------+
| ``if (obj.attr("foo"))``              | ``if (py::hasattr(obj, "foo"))``            |
+---------------------------------------+---------------------------------------------+
| ``if (obj["bar"])``                   | ``if (obj.contains("bar"))``                |
+---------------------------------------+---------------------------------------------+