reset password
Author Message
asalin14
Posts: 12
Posted 11:22 Mar 25, 2019 |

In class, I mentioned an issue with extending type in Python.

You would think that if a class extends type, it would take all of its functions. We can see this is partially true.

 

According to the type documentation, the type class' constructor can do two things:

One Argument:

When running type(one_arg), type usually returns the .__class__ value of that argument.

Three Arguments:

When running type with 3 arguments, you must include (class_name, bases, dict). This is what is ran when creating a new class with metaclass type. A valid example call would be type('Squre', (Shape), {length: 1}).
 

You would think extending type would allow you to run both of these functions but Python only allows you to run this with three parameters. I have attached a Python file showing this interaction.

 

In the type documentation (https://docs.python.org/3/library/functions.html?highlight=type#type), it addresses this issue:

Changed in version 3.6: Subclasses of type which don’t override type.__new__ may no longer use the one-argument form to get the type of an object.

 

Does anyone have any ideas on why this change might have been made? I suspect that type was implemented in C. I don't think it is possible to restrict inheriting functions when implementing functions in Python. I think it would help to find the implementation in the Python source but I'm not sure where to start.

 

Attachments:
rabbott
Posts: 1649
Posted 12:01 Mar 25, 2019 |

Your attachment has this code. See added comments. 

class Road(type):
  pass

class X():
  pass

# This calls the function that creates a class named 'Hello'.
print(type('Hello', (), {})) # => <class '__main__.Hello'>

# The following line is equivalent to the preceding line because Road
# is a subclass of (the class) type. So, since Road is not defined as a function 
# the superclass function (type) is called. (At least I believe that's what's happening.)
print(Road('Hello', (), {})) # Equivalent to type('Hello', (), {})

# The following calls the function that asks for the type of X. It
# doesn't really call type as a class.
print(type(X))
# The preceding is as if we had a function
def type(x): 
   return x.__class__
# and we are calling that function. We are simply asking to look up the
# __class__ attribute of x.

# The following line asks to create an instance of Road. But since Road is
# itself a type, it is asking for an instance of type. To create an instance
# of type requires the function type(_, _, _), which calls type.__new__, which
# takes three arguments.
print(Road(X)) # => TypeError: type.__new__() takes exactly 3 arguments (1 given)
Last edited by rabbott at 12:28 Mar 25, 2019.
asalin14
Posts: 12
Posted 12:53 Mar 25, 2019 |

Both are calling the same __new__ method of type/Road. (Not __init__ as I claimed earlier)

Attachments:
Last edited by asalin14 at 13:00 Mar 25, 2019.
rabbott
Posts: 1649
Posted 16:57 Mar 25, 2019 |

Another bit of information.

>>> help(type)
Help on class type in module builtins:
class type(object)
 |  type(object_or_name, bases, dict)
 |  type(object) -> the object's type         (a)
 |  type(name, bases, dict) -> a new type     (b)
 |  
...

My guess is that this means that type can be called in two ways,
(a) either with one argument (an object), in which case it returns the type of its argument,
(b) or with three arguments (a name, a tuple of bases, and a dictionary), in which case it creates a new type, i.e., a new Class. 

In other words, depending on how type( ... ) is called, two completely different functions are executed.

As Abel pointed out, this is not consistent with standard Python syntax. Normally one can't use the same name for two functions, even if the two functions have different signatures.

It's been a while since I used Java, but if I'm remembering correctly, one can use the same name for different functions in Java. How about C/C++ ? Since Python is implemented in C++ (or is it straight C?) can use use the same name for multiple functions in C?

Last edited by rabbott at 17:00 Mar 25, 2019.
asalin14
Posts: 12
Posted 18:09 Mar 25, 2019 |

I finally found the type.__new__ implementation (type_new):

https://github.com/python/cpython/blob/3.7/Objects/typeobject.c#L2349

 

The issue is addressed:

/* Special case: type(x) should return x->ob_type */

/* We only want type itself to accept the one-argument form (#27157) Note: We don't call PyType_CheckExact as that also allows subclasses */

In the code, we can see that it differentiates the different functions (type(X) and type(str, bases, dict)) by counting the number of arguments. It only allows the one argument when the type passed in is exactly a PyType_Type. 

 

The reason for this implementation is explained in this link:
https://bugs.python.org/issue27157
 

My understanding is that checking if the passed in class is a subclass of type every time the type function is called is a waste of resources. Since type is called any time a class object is created, Python could be more efficient by only accepting the one parameter call when the type is exactly type (since it would not have to also check if it is a subclass of type).

Last edited by asalin14 at 18:11 Mar 25, 2019.
rabbott
Posts: 1649
Posted 18:18 Mar 25, 2019 |

This is getting a bit involved (and probably less and less interesting), but here are a few more examples.

class Road(type):
  pass

class X:
  pass

class W:
  pass

x_object = X()
y = x_object.__new__(W)
print(type(y))                            # => <class '__main__.W'>


z = 1.0.__new__(float, 42)
print(z, type(z))                         # =>
 42.0 <class 'float'>


print(type.__new__ is Road.__new__)       # => True
print(type.__new__ is W.__new__)          # => False
print(x_object.__new__ is W.__new__)      # => True
print(bool.__new__ is W.__new__)          # => False
print(object.__new__ is x_object.__new__) # => True

 

Last edited by rabbott at 18:24 Mar 25, 2019.