-
Notifications
You must be signed in to change notification settings - Fork 17.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
proposal: type parameters are comparable unless they exclude comparable types #52614
Comments
Sorry I must be misunderstanding something, but if you were to instantiate a generic function with a concrete non comparable type (e.g a slice) wouldn’t you consider the type parameter to not be an interface? The point that type parameters are interfaces is confusing to me. Also couldn’t this cause weird action at a distance? A deeply nested generic function requires comparable type parameters and you don’t realize it? I realize I’m not fully read up on this issue and there’s a lot of history here, but the proposed behavior feels weird and surprising as a dumb user, which is concerning me. |
@hherman1 The view of a type parameter as an interface mostly affects type checking: for instance, in a generic function, the type of a type parameter is unknown when that function is type-checked (consider a generic exported library function were one knows nothing about clients - still we want to fully type-check that function). For type-checking we therefore treat the type parameter (more precisely, a variable of type parameter type) as an interface in the sense that we have to consider all possible types in its type set (we know a bit more than that, for instance we know that the type of such a variable, even if unknown, doesn't change for the duration of that generic function, which is why these are somewhat special interface types). It's also this type set we care about when that type parameter is used to instantiate yet another generic function. We already have "weird action at a distance" with non-generic Go: a function operating on ordinary (non-type parameter) interfaces may hold a dynamic type that doesn't support comparison yet that function may try to compare the interface (all ordinary interfaces support |
I think it could work. It still feels like we are admitting defeat on having full static checking. If we can use the provision, why not just make type parameters able to take a list of constraints? type Set[T (any, comparable), V any]map[T]V ? |
Does this proposal mean the following code will become valid? func foo[T any](x T) {
_ = x == x
}
Is there still a |
Just to be clear, this program func eq[T any](a, b T) bool { return a == b }
func main() {
var s []int
eq[[]int](s, s)
} would be defined to always panic at runtime, right? Not a compile error even though it's statically obvious that it can't succeed? If the compiler decides to stencil func(a, b []int) bool { panic("[]int cannot be compared") } since there is no way to implement the comparison. How does this interact with comparisons to func isnil[T any](a T) bool { return a == nil } would still be invalid, since |
@atdiar I don't understand what you're asking about with the "list of constraints" @go101 Yes, the func foo[T any](x T) {
_ = x == x
} would become valid since the type set of @magical Correct, your example calling |
@griesemer It's relevant to that proposal in the sense that it would avoid the use of a static tool. If it's really too off-topic I will disengage but I am curious. |
I wonder how feasible to remove |
To me, this throws the baby out with the bathwater. It seems to have ~all of the downsides of #52509 except that it converts some compile time errors into run time panics. I assume the argument is that it is confusing to have compiler errors for some bugs of this class of, but not all - but IMO that is completely fine and not a good argument to not have compiler errors for any bugs of this class. I still believe both #51338 and #52509 are acceptable solutions. And the more we talk about it, the more I become entrenched into my view that they are the best ones (either of them). They have downside, but it's fine to accept downsides sometimes.
IMO, if we do this, we should absolutely remove In other words, one of the flagship use cases for |
How does this definition differ from "Types that supports == or !="? Specifically, what's the differentiation between this proposal and #52509? |
@griesemer thanks for taking the time to write this up more concretely, I still think this would be a very good simplification for the language as a whole (and agree with @Merovius that assuming this proposal is accepted, deprecating To the point around bathwater... I do agree that it reduces some compiler completeness; but I don't think the programmer error of "passing something not comparable where comparability is needed" is a very common one in my experience (compared to say, just a logic bug). As pointed out in the proposal, it would still be found by cursory testing. So I think the trade-off of "a very confusing keyword" vs "a slightly longer turn around time to identify specific classes of bug in your code" is well worth it overall. @changkun That proposal still leaves |
Thanks for clarifying this. The goal here is radical as we already have Take a step back, and I think the suggested wording in CL 401874 that separates the notion of interface and type set may balance this better. As noted in the CL commit message: Defining "a type parameter is an interface", "may reflect a simplification, but it is also cumbersome for the predeclared identifier
This observation points to the key challenge of the current generics design: Type constraints are limited by the expressiveness of an interface, there are (more than one possible) type sets cannot be defined using an interface. This should arise the reflect: As previous discussions revealed the need of multiple differently defined comparable definitions (in real world), it might worth to do the separation between interfaces and type sets. |
I don't think it poses a problem, but if we ever make all interfaces into full types, this restriction would fully go away, as you could always instantiate a generic function using its constraint. i.e. the set of types a constraint would allow will always include a comparable type - the constraint itself. It's not a problem, because it will be a relaxation of a restriction (i.e. it will only allow more programs to compile, not prevent previously compiling programs to stop compiling). It might hint at a bit of strangeness about this rule though.
I don't think #52509 leaves us with "a very confusing keyword" (nit: predeclared identifier) at all. So I disagree that this is a tradeoff we are making. In particular, anything that would still be confusing about IMO #52509 even makes the better tradeoff - because when that happens, there is at least a very clear indication in the API, that the values are supposed to be comparable. That's not something we even have here. That is, with #52509, if you get a panic by instantiating a generic function using an interface type and then get a panic in the comparison, you can at least look at the signature and say "oh, it says the value needs to be comparable, makes sense", whereas here, you end at "well, the signature says |
Just leaving that as a note but I agree with @changkun #52614 (comment) The problem with That's the reason behind #52531: to be able to introduce a refinement so that we can select the comparable (in its spec sense) subset of a set of types. Although it might be more complex in implementation, I don't know why it was deemed as not addressing the issue. Also a little nit, it seems to me that the set of permissible types is not always a type set. For instance, basic interfaces are included in the set of permissible types, not in their own type set. |
I wouldn't say that it doesn't address the issue. I would say that it's either a) the same as #49587 (in its original phrasing) or b) the same as #52509 (with your latest comment, after removing the second comparison constraint). It is phrased more complicatedly, but in terms of what code it allows to write and how that code is written, it does the same. Both of those proposals do address the issue. Both have other downsides. But that doesn't mean we can't accept them anyways. I wouldn't say there has been a final decision made on any of these proposals. |
Well if you leave the part about But yes, on the topic of the current proposal, it feels like we are dropping the ball perhaps too early. Although it could be seen as an engineering tradeoff, I'm not too sure about it. 😕 |
I think the only practical difference this makes, is that it doesn't allow mixing |
I don't want to hijack this issue much more but there is still a practical difference. |
What happens to derived map types?
does this become valid? Is instantiating it with a non-comparable type a typechecker error or a runtime panic? Or can you actually get a |
IIUC, that would panic when run. Here is another tough case. This does not compile: func f(a []byte) {
var b []byte = nil
_ = a == b
} But presumably this will compile and panic at runtime? func f[T *any|[]byte](a T) {
var b T = nil
_ = a == b
}
f[[]byte](nil) It's confusing because you can compare []byte to nil, but only against a constant nil, not against a variable that happens to have nil in it. |
An alternative here is to adjust the meaning of "comparable" to specifically exclude interfaces that have no comparable types. At least to me, that rule seems at first like it is a bit too subtle, but we already have precedent with struct and array types that the comparability of a type can depend on the contents of the type definition. It is a straightforward rule which also seems reasonable to include in the spec change for this proposal. |
I said this in more words in one of the previous proposals, but it seems more relevant here and so I'll state it again more concisely this time: My general assumption is that whenever I use interface types (by which I mean: the pre-type-parameters mechanism for opting in to dynamic dispatch based on an interface) I am opting in to the possibility of panics at runtime in return for the flexibility of choosing a concrete type at runtime. Therefore I expect to be able to choose to place an interface type in a type parameter and have the compiler defer to runtime any checks that depend on the final concrete type. This is under control of the caller of the generic function/type and so their own risk to take; the author of the callee only needs to specify that they require a comparable type, without regard to whether a caller will satisfy that constraint statically (by using a concrete type that the compiler can prove is comparable) or dynamically (by using an interface type where comparability is known only at runtime, and might therefore panic). Constraining use of interface types because we don't know what concrete type they will contain at runtime rather seems to defeat the benefit of having them in the language. Callers are still free to use concrete types if they want the additional static guarantee of no panics at runtime. This proposal seems to be a compromise within that mental model and so it seems favorable to me. |
That's interesting. Why is that? Specifically, would the following code panic? (one could replace
|
@atdiar
|
One way to get around the problem with derived map types is to only allow a type parameter For example This would allow comparability to be written as a constraint without having the weird paradoxical As for the equality operator And There are a few problems with this:
|
[edit] I think we understand the issue differently. From my understanding of your explanation, P is a variable of type What does not seem to work is that traditionally, for interface variables, the values that can be assigned to it are any value that implements the interface. This is not always the case for type parameters. That's why the definition of interface implementation does not exactly match here: it's a necessary but not sufficient condition. (not an equivalence) |
The map edge-case really comes down to ‘when’ it should fail. I think the clearest answer is that it should fail as though the map had been instantiated with the key type of These two snippets should both compile and both fail with a runtime error when setting a key (assuming the runtime type of ‘c’ is not comparable).
Although it would be possible to panic when creating an instance of such a map, it seems better that these two functions fail equivalently instead of introducing a new failure mode to the language. The simplification this gives you (in terms of programmer mental model) is that the behavior of code that compiles successfully depends only on the runtime type of values; you don’t have to think about separate failure modes for different compile time type arguments. |
Let me repeat this clarifying question, then. And would you agree, that the logical conclusion of this is that we should abandon the concept of static comparability altogether and allow |
That panic will only occur in a determinate manner if we do not have any branching in the body of the function or in the functions that call it. |
The problem with |
FWIW I think it would absolutely be possible to accept that all comparisons are valid and comparisons (and usage as a map key) of any I don't see a lot of middle ground though. I don't think we can allow comparisons and usage as map key for all type-parameters, while not also allowing them in non-generic code - because once we admit that these types can exist in some circumstances, carving out room so they don't exist in all circumstances seems complicated and confusing. Another strange program, BTW: func F[T any]() func(T, T) bool {
return func(a, b T) bool { return a == b }
}
func main() {
a := []byte{}
f := F[[]byte]
f(a, a) // panic, presumably
} I think it would be very strange to adopt this proposal while not extending comparability to all types in general. |
In the example, the type of
In other words, the fact that To repeat: Interface implementation is constraint satisfaction.
Sure. All I am saying is that for purposes of type rules (and thus type checking),
I couldn't disagree more. The fact that a constraint is an interface is what makes it all work. It's the insight that got us from a vague notion of "constraints" in 2018 to a thorough understanding of what a constraint is in 2021 and propelled the generics effort forward. It's the crux behind the type theory of Go generics. The problem we're dealing with is not that constraints are modeled as interfaces or that constraint satisfaction is interface implementation. The problem is that ordinary Go interfaces have an |
It sounds like if we had Which seems like a case that we should accept #51338 and a vet check, if Are there any fundamental problem with that, apart from the virality-concerns of |
@Merovius Exactly. See also the last paragraph in #50646 (comment). I think you are probably correct that if we accept this proposal, we have to make But it comes with a loss of static type safety that we may not be willing to accept for generic code. (The compiler could probably still complain if we directly compare concrete slice or function types, but that's beside the point.) If we want static type safety for I think there's no middle ground here: we either a) make I also agree that accepting #51338 is probably the only viable alternative, the flip side of this coin. (The At the moment I also don't see any problems with #51338 besides the virality concerns. |
@Merovius re: "And would you agree, that the logical conclusion of this is that we should abandon the concept of static comparability altogether and allow == to be applied to any type and panic, if the type is inappropriate (just as if all values where wrapped in any)?" Not at all. I do think having fewer non-comparable values would be good for Go; but that is a separate discussion. The way I think about it, is that there are three different "time"s at which an error can be caught:
I would suggest that an important property to preserve is that the "compile time when code is used" behavior depends only on the type-signature of the function, not on its implementation. (I think that is the case today, and under #52509, and under this proposal). I don't think we need to go so far as to say "any code that uses == should compile". An emergent property of that (in combination with The other emergent property (that I really like) is that you can "inline" any method set type parameter to get a specialized function with the same behavior.
For #52509 the emergent property is that you need to specify comparable everywhere, despite it not really adding any additional safety.
I think this is not really adding any safety because (in my experience) you are most often in the "runtime error" camp anyway; and so having two ways to fail what would otherwise be very similar pieces of code seems unnecessary to me. #51338 does also have the emergent specialization feature
which is nice; but continues to fail to compile programs that would run just fine, and it's not obvious how to fix them.
This is (I think) a matter of taste, more than anything else. I really enjoy that go generics re-use interfaces, and I think that was a great insight to simplify things and avoid adding new concepts. I also really enjoy a language that doesn't require too much ceremony, and statically declaring comparable or not seems (to me) like too much ceremony. |
The way I see it, we risk complicating either interfaces or constraints or both. I think I see what you mean by a type parameter's underlying type is an interface but I still don't see how P could be instantiated by If we accept that a constraint defines the set of permissible type arguments, I don't understand why the only way to construct such a set has to be an interface. Anyway, that's my view, I guess we will see, but I find it quite a pity that all constraints have to be an interface. This is too inflexible imho. That interfaces can be constraints is fine. This is just a way to denote a set of types. |
This is the current strategy for (almost?) all generics implementations except C++. It is also current Go strategy for every non-generic code. You can't call do stuff on type which it does not support explicitly.
This is C++ strategy and it's very bad if you have layers of generic code, because it produces error messages deep down the line. In Go case vet check will only catch very simple errors, because traversing entire tree is unpractical.
I'm not sure what you mean. |
The discussion about whether interfaces and constraints are exactly the same thing or whether one is a subset of the other etc is an interesting way to draw out the details of how exactly this could work. The main way I've been thinkiing about these proposals focused on I bring this up because I see the discussion shifting to the idea of |
I'm not sure. You obviously can't unless the constraint is a basic interface. But I also don't think it's strictly true even if it is. Here's a dumb example: func F[T any](v T) { fmt.Printf("%T\n", &v) }
func G(v any) { fmt.Printf("%T\n", &v) }
func main() {
F(42) // prints "*int"
G(42) // prints "*interface{}"
} Maybe I'm picking nits. But it also also definitely would be not true for writing out a specific instantiation, which I think is similarly important.
No, the second one would be a compile time error as well. The compiler would infer But no matter. #52509 has other problems as well.
Over the long term, hopefully, by being deliberate about embedding FTR your specific example is, IMO, a bad one. I don't think
I think |
We definitely risk complicating interfaces or constraints or both if we separate constraints from interfaces as you seem to hint at. Which is why we don't.
It does exactly denote an option between string or int. And of course any interface that implements it which again could only contain string or int. Seems pretty consistent. Rather similar to a bool variable which can be assigned the value true or false, and any bool variable which again could only hold a true or false value.
A type parameter constrained by interface{string | int} definitely has a "+" operator. And if we were to allow such interfaces as ordinary variable types we could make the "+" operator work by dynamically checking that the dynamic types match (and panic otherwise). That seems consistent with the dynamic dispatch we do for methods. And it matches exactly the dynamic check we do for
So you're suggesting a separate language construct which is a type set, used as a constraint, and which somehow is different from an interface. This is similar to what others have suggested in #52509. But such a type set would have to interoperate with concrete and interface types somehow. Specifically, we'd need to explain how an interface implements a type set (constraint) and how a type set implements an interface (when a type parameter is assigned to an interface); maybe even the same interface. See also #52509 (comment) which I believe addresses the same question. I'd love to see the exact rules of how interfaces and such hypothetical constraint type sets interoperate. But it's hard to imagine that this would be simpler (and more flexible) than what we have now.
The beauty is that comparable is a constraint, because interfaces and constraints are the same. Besides, how would it solve the issue with type sets and virality "just as well"? See again the previous paragraph on how constraints and interfaces must work together.
See the previous paragraph and also my previous reply (last paragraph). |
Not really suggesting a type set because type sets do not include interface types. The explanation about unions is a complication of interfaces imho. That they might inherit possibly panicking operators from their type sets is more complex. But fair enough. That Differentiating Also because comparable would not be an interface, interface types would not have to implement it. But if all interfaces have to implement We can see this problem because f is in the typeset of
We just need to check that types have the comparison operators to satisfy the constraint. This is not something that looks difficult to do, unless I am mistaken. |
Experience report: Here is a blog post where Tim Bray (ex-AWS) is confused about |
It seems pretty clear at this point that this proposal, at least in the current form, is not providing the kinds of static guarantees that people would like to ensure. Retracted. |
This proposal has been declined as retracted. |
Introduction
The predeclared type constraint
comparable
, introduced with Go 1.18, is a (magic) interface describing the set of types for which==
is expected to work without panic. The introduction ofcomparable
led to considerable discussion (see background in #52474 for details). It also led to confusion because the set of types described bycomparable
does not match the types that are considered comparable per the Go spec.Here's the current list of issues related to this discussion:
comparable
#50646any
no longer implementscomparable
#51257comparable
type parameters #52509The goal of these proposals is to address the perceived shortcomings of
comparable
by changing its definition or by separating the notion of interfaces and type sets.So far none of these proposals (if still open) have gained significant traction, and none of them directly address the core of the
comparable
problem: in Go ordinary interfaces are always comparable, i.e., they support==
and!=
independently of whether the dynamic type of the interface is comparable. We cannot change this without breaking backward-compatibility.Instead we propose to embrace this property of interfaces.
Proposal
The underlying type of a type parameter is its type constraint interface; i.e., a type parameter is an interface (albeit with a "fixed" dynamic type which is given when the type parameter is instantiated). Because type parameters are interfaces, we propose:
Type parameters are comparable unless the type parameter's type set contains only non-comparable types.
This is the entire proposal.
Discussion
The reason for having
comparable
in the first place is to be able to statically express that==
is expected to work and that it won't panic. If this proposal is accepted,==
will be supported on type parameters unless the type set contains only non-comparable types. We will also lose the guarantee that==
won't panic (if==
is supported in the first place). We may still keepcomparable
, but more on that below.This proposal hinges on the premise that losing the static "no-panic" guarantee is not as severe a loss as it might appear at first. We believe this could be true for the following reasons:
We are well-accustomed to the fact that
==
on ordinary interface types might panic. In code, we tend to address the comparability requirement through documentation; we suggest that we continue to use documentation for this.If a type parameter is instantiated with a non-comparable type and
==
is expected to work, upon invocation the generic code is likely to panic right away. This contrasts favorably to the situation with ordinary interfaces where a panic may occur for some of the dynamic values but not all of them. In other words, making a comparability mistake in generic code would be detected quickly, probably in the first test run.Better yet, we don't have to rely entirely on dynamic type safety: it should be straight-forward to introduce a
vet
check that reports when a type parameter for which we expect==
to work is instantiated with a type that is not comparable. Such a check would provide the equivalent of a static compile-time check, and virtually eliminate the risk of==
-related panics.With this proposal unfortunate restrictions caused by the use of
comparable
can be avoided. The==
operator will simply be available to all type parameters unless their type sets contain only non-comparable types (it makes sense to exclude such type sets because we know with certainty that==
will always panic for such type parameters). Examples:This proposal also opens the door to more flexible (if perhaps esoteric) generic code that relies on
==
to work for some type instantiations but not for others, something that can be readily expressed through control flow but which is much harder (or impossible) to encode through types.We still have the option to keep
comparable
as the "umbrella" set of types which are comparable without panic. Or we could decide to remove it because using it may preclude some uses of generic code (e.g., see #51338 (comment)). Keeping it will also require a programmer to always make the decision whether or not to use it. To remove it we could make use of the provision in the Go 1 compatibility guarantee:Eliminating
comparable
would simplify the language and probably eliminate some confusion. The decision whether to keep or remove it is independent of this proposal.History and credits
We briefly toyed with a simpler form of this idea (type parameters should always be comparable) as a potential solution to the
comparable
problem shortly before the 1.18 release. At that time we dismissed making all type parameters comparable (and eliminating the predeclared typecomparable
) as too radical. The resulting loss of static type safety around==
in generic code seemed unacceptable.We are aware of at least one other person, Conrad Irwin, who independently suggested that all type parameters should be comparable in #52509 (comment).
The text was updated successfully, but these errors were encountered: