-
Notifications
You must be signed in to change notification settings - Fork 102
CMP-7543 Retain AnnotatedString in ClipEntries on Desktop #2092
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CMP-7543 Retain AnnotatedString in ClipEntries on Desktop #2092
Conversation
| DataFlavor(AnnotatedString::class.java, "AnnotatedString") | ||
|
|
||
| @Suppress("UnusedReceiverParameter") | ||
| val DataFlavor.annotatedStringFlavor: DataFlavor |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A public API change require running ./gradlew :mpp:jbApiDump and commiting the change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cc @igordmn, Igor adding you as a reviewer as here we have an API change for desktop. (Also there was a discussion in a slack thread - https://jetbrains.slack.com/archives/C076VK0LGH1/p1746625562243429)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done, API dump updated!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For context, this is necessary for users to be able to use the Transferable#isDataFlavorSupported(DataFlavor) and Transferable#getTransferData(DataFlavor) APIs. Otherwise, they'd have to guess what the flavour to use is...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I gave it another thought.
IMO, It would be better to move a new public property to ui module, in PlatformClipboard.desktop.kt, so the new property won't expose an extra public class ClipboardUtils_desktopKt (we can see it now in foundation.api dump).
And adding @ExperimentalComposeUiApi to it would be safer for now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is easy to break Jewel and any other users when there is no actual API that you're changing/breaking
It will be a definite breakage after CMP-7558, as the main purpose of it is to not expose implementation details to the external users.
So, is the main use case is to retrieve the annotated string from ClipEntry? If so, then my suggestion of removing won't help your case if you still need creating DataFlavor("AnnotatedString").
Considering the AWT coupling issue, the best way to solve this case is to have public functions that provide AnnotatedString. Similar to the internal ones:

It is a big task, which should be done in AOSP first as common API. There are probably alternatives, but before suggesting one, I would like to understand the main issue more.
After this PR we don't lose information in the standard copy/paste actions anymore. Why we still need a custom clipboard and retrieve the annotated string from the default one? Suppose, we copied a text from Text/BasicTextField, and Compose created an AnnotatedString and stored it in the default clipboard. Where we need to retrieve it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It will be a definite breakage after CMP-7558, as the main purpose of it is to not expose implementation details to the external users.
We can change the Jewel implementation (we'll keep it internal) when CMP-7558 ships and we get it in Jewel, that is not a huge problem.
So, is the main use case is to retrieve the annotated string from ClipEntry? If so, then my suggestion of removing won't help your case if you still need creating DataFlavor("AnnotatedString").
Yes it is, but the main issue this PR fixes is that right now the AnnotatedString that gets passed to Clipboard is stripped of all annotations. I can live with having to have my own copy of the DataFlavor until CMP-7558 is done, and we can use whatever better API there will be.
Considering the AWT coupling issue, the best way to solve this case is to have public functions that provide AnnotatedString. Similar to the internal ones:
[...]
It is a big task, which should be done in AOSP first as common API. There are probably alternatives, but before suggesting one, I would like to understand the main issue more.
Agreed, that would be the ideal scenario, but I was told we could not add any new public API, especially in commonMain. In the meantime though this is a way (undocumented, unofficial) that still allows Jewel to fix a pretty bad UX when copying rich text.
After this PR we don't lose information in the standard copy/paste actions anymore. Why we still need a custom clipboard and retrieve the annotated string from the default one? Suppose, we copied a text from Text/BasicTextField, and Compose created an AnnotatedString and stored it in the default clipboard. Where we need to retrieve it?
Jewel will implement its own Clipboard (see this PR for example) where it will take the AnnotatedString and convert it to raw Markdown, which is a format that can be then put in the AWT clipboard. This is necessary, because we can't put non-serializable content in the system clipboard. It is also what IJ does in general when you copy some highlighted code to the clipboard — it has a mechanism to "convert" said rich text to HTML and RTF, so it can preserve its formatting when pasted elsewhere.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If it is okay that it will definitely break, we can merge it as is, though, let's add a comment for private val annotatedStringFlavor: DataFlavor:
This implementation detail is used by Jewel. When removing it, please provide an alternative of retrieving an annotated string, and notify a Jewel developer that they need to change the implementation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After https://youtrack.jetbrains.com/issue/CMP-7558/New-Clipboard-Adoption.-Review-API or https://youtrack.jetbrains.com/issue/CMP-7624/Design-ClipEntry-common-API there should be a way to retrieve the annotated string: defaultClipEntryFactory.asAnnotatedString(clipEntry), clipEntry.asAnnotatedString().
Something like awtAnnotatedStringFlavor: DataFlavor probably won't be exposed as API for desktop, as it is only one of the many ways to provide it. Instead of putting the annotated string to transferable, we could put our own class with metadata, or even don't create Transferable until it is needed for passing to the clipboard.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If it is okay that it will definitely break, we can merge it as is, though, let's add a comment for
private val annotatedStringFlavor: DataFlavor
Done
After https://youtrack.jetbrains.com/issue/CMP-7558/New-Clipboard-Adoption.-Review-API or https://youtrack.jetbrains.com/issue/CMP-7624/Design-ClipEntry-common-API there should be a way to retrieve the annotated string: defaultClipEntryFactory.asAnnotatedString(clipEntry), clipEntry.asAnnotatedString().
Something like awtAnnotatedStringFlavor: DataFlavor probably won't be exposed as API for desktop, as it is only one of the many ways to provide it. Instead of putting the annotated string to transferable, we could put our own class with metadata, or even don't create Transferable until it is needed for passing to the clipboard.
That's fine. As long as we still have access to the full AnnotatedString in Clipboard via whatever API ends up being there, we are ok with that.
150f8b6 to
7f24fbc
Compare
| DataFlavor(AnnotatedString::class.java, "AnnotatedString") | ||
|
|
||
| @Suppress("UnusedReceiverParameter") | ||
| val DataFlavor.annotatedStringFlavor: DataFlavor |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done, API dump updated!
| DataFlavor(AnnotatedString::class.java, "AnnotatedString") | ||
|
|
||
| @Suppress("UnusedReceiverParameter") | ||
| val DataFlavor.annotatedStringFlavor: DataFlavor |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For context, this is necessary for users to be able to use the Transferable#isDataFlavorSupported(DataFlavor) and Transferable#getTransferData(DataFlavor) APIs. Otherwise, they'd have to guess what the flavour to use is...
| // TODO: https://youtrack.jetbrains.com/issue/CMP-7543/Support-styled-text-in-PlatformClipboard-implementations | ||
| internal actual suspend fun ClipEntry.readAnnotatedString(): AnnotatedString? { | ||
| return readText()?.let { AnnotatedString(it) } | ||
| if (!hasAnnotatedString()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This check is done to retain the existing behaviour in androidx.compose.foundation.text.selection.TextFieldSelectionManager#paste, which only tries to get an AnnotatedString:
val text = clipboard?.getClipEntry()?.readAnnotatedString() ?: return@launchI will propose to Jetpack Compose to alter this slightly as:
val text = clipboard?.getClipEntry()?.readAnnotatedString()
?: clipboard?.getClipEntry()?.readText()?.let { AnnotatedString(it) }
?: return@launchTo make it more explicit at the usage site, but for now this covers that case.
| } | ||
|
|
||
| override fun lostOwnership(clipboard: Clipboard?, contents: Transferable?) { | ||
| // Empty |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know why StringSelection implements ClipboardOwner only to have its only function do nothing; I however am going to retain the same behaviour, just in case it covers some unintuitive edge cases. It shouldn't cause any issues, judging by the JavaDoc of ClipboardOwner.
| } | ||
|
|
||
| companion object { | ||
| private val supportedFlavors = arrayOf(_annotatedStringFlavor, DataFlavor.stringFlavor) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note: when AWT puts this transferable in the clipboard, it drops the AnnotatedString flavour, since it's not serializable. The net result is having just the plain text put in the system clipboard, which is consistent with the current behaviour.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it drops the AnnotatedString flavour, since it's not serializable.
What if I have this code in the same app (ignoring the fact I use internal functions here):
val annotatedString = AnnotatedString(...)
val clipEntry = annotatedString.toClipEntry()!!
clipboard.setClipEntry(clipEntry)
...
val result = clipboard.getClipEntry().readAnnotatedString()is result expected to be equal to annotatedString ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it's what one of the tests is checking, too.
In prod code however the clip entry is handed immediately to the Clipboard, which in turns sets it immediately in the AWT clipboard, which drops the annotated string as described above. So, the only time the annotated string really "exists" in the transferable is while in transit via the Compose clipboard. As soon as it's handed over to AWT it is discarded.
| } | ||
| val typeface = when (font) { | ||
| is ResourceFont -> typefaceResource(font.name) | ||
| // TODO: replace with FontMgr.makeFromFile(font.file.toString()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I removed this TODO I left here a couple years ago as it seems it was addressed in the line below. Someone just forgot to remove the TODO :)
| transferable ?: EmptyTransferable, | ||
| null | ||
| /* contents = */ transferable ?: EmptyTransferable, | ||
| /* owner = */ transferable as? ClipboardOwner, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this was technically wrong before, as we are supposed to provide the owner to the AWT clipboard if there is one. Given the Transferable itself is the only way we could have an owner, if it implements ClipboardOwner we pass it as owner.
ClipboardOwner is a simple interface that only provides a callback to know "you're not the owner of the clipboard any longer", meaning something else was put in the clipboard. I think this can be needed to do some cleanup in some Transferables which may be holding on to resources internally.
...dation/src/desktopMain/kotlin/androidx/compose/foundation/internal/ClipboardUtils.desktop.kt
Outdated
Show resolved
Hide resolved
ce7e4c9 to
4a22adf
Compare
...dation/src/desktopMain/kotlin/androidx/compose/foundation/internal/ClipboardUtils.desktop.kt
Show resolved
Hide resolved
4a22adf to
45a28c9
Compare
|
I'd like for Igor to weigh in on this before merging. It feels somewhat of a hack that is only useful because Locally, I don't like |
This is in any case a necessary change, even if you wanted to also figure out a way to serialize and deserialize the Yes, it unblocks Jewel being able to access the
We could, that doesn’t really change the substance, though, and still requires manually adjusting the mime type to also carry the information about the original data class. For example, where IJ uses it, it does something like: // PsiCopyPasteManager
ourDataFlavor = new DataFlavor(DataFlavor.javaJVMLocalObjectMimeType + ";class=" + flavorClass.getName());Which to me feels like even more of a hack. It also has the same drawback of requiring users to know how to conjure up the right mime type, unless we provide a |
|
I guess I don't understand why we need to expose the |
|
We don't expose the fact that plain text goes into the clipboard with |
|
Because y'all have aliased the native APIs to the AWT ones, and it's not like there are many alternatives to that... I can't think of any, at least. If you have alternative ideas I am happy to hear them. Would making |
Due to a lack of any real API to deal with ClipEntries, desktop users who want to access that need to know that The difference is that they have those flavour constants defined in AWT already, while there isn't one defined for We can remove the new data flavor and I can just have it in Jewel, but at that point we're really just obfuscating something for the sake of obfuscation. |
| internal fun ClipEntry?.hasAnnotatedString(): Boolean { | ||
| if (this == null) return false | ||
| val transferable = nativeClipEntry as? Transferable ?: return false | ||
| return transferable.isDataFlavorSupported(annotatedStringFlavor) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the transferable is plain text, we should probably return true here as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not done on purpose, because if you do, you have no way to distinguish the case where there is plain text, and there is no AnnotatedString. It doesn't happen when we put things in the clipboard, but nothing prevents users from putting whatever they want in the clipboard, including a situation like this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure you need to distinguish that. If you're expecting to receive an annotated string, you call has/readAnnotatedString and that's it.
As-is, what we have is inconsistent: hasAnnotatedString can return false and then readAnnotatedString returns a non-null value.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also to note, we're assuming that the plain text and annotated string contents are interchangeable as at most the plain text is annotatedString.text. Compose does, in general. In normal cases, they are. But here again, absolutely nothing says they must be. You might have a different semantic representation of the same content between plain text and AnnotatedString.
An example; an app puts this text as AnnotatedString: "Hello world". But, for some reason that makes sense to them, the plain text content is Hello *world* — the app is expressing the bold part in the plain text as asterisks, to maintain the intent, even though the representations now have diverged. This is a perfectly valid use case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure you need to distinguish that. If you're expecting to receive an annotated string, you call has/readAnnotatedString and that's it.
These are internal APIs. hasAnnotatedString isn't even an expect/actual, it's literally only used in the desktop target. Users can't access these.
As-is, what we have is inconsistent: hasAnnotatedString can return false and then readAnnotatedString returns a non-null value.
This is only needed because @eymar said that we shouldn't change the commonMain code that uses these APIs (it only uses read* APIs), but this works around an issue in the implementation that only ever calls readAnnotatedString and never falls back to readText. I think you can find that conversation in some other comments to this PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
|
The reason there's no normal API to put/get clipboard data is that we haven't implemented it yet. |
|
If the only thing that's stopping this PR from going in is the public flavour, I can remove it. Until there is a proper API we can use, I'll keep a copy of the flavour in Jewel and use it there. Not the end of the world for me. |
|
I think my opposition to this is due to the fact that I haven't yet thought about what we want to do with the clipboard API, so I don't know whether this change is aligned with that. But if this improves the situation for jewel, without creating problems for us down the line, I'm not against it. I would still like for Igor to look at it, though. |
| DataFlavor(AnnotatedString::class.java, "AnnotatedString") | ||
|
|
||
| @Suppress("UnusedReceiverParameter") | ||
| val DataFlavor.annotatedStringFlavor: DataFlavor |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For context, this is necessary for users to be able to use the Transferable#isDataFlavorSupported(DataFlavor) and Transferable#getTransferData(DataFlavor) APIs.
Could you elaborate more, and provide code how it should be used in outside code?
If it is not for the outside code, I would suggest to remove it as public API. The AWT implementation should be only one of many possible implementations.
Detailed explanation
There is an architectural issue in the current API in jb-main that the Clipboard API is tightly coupled with AWT.
It is tightly coupled because it creates AWT-based classes in one place (in toClipEntry), and uses casting to AWT classes in another place (in readAnnotatedString/readText). Instead of that we had to keep ClipEntry platform-independent, and store raw data in it, not the AWT based objects. Convert to AWT based class only:
- in places where it is needed - in the copy/paste actions itself, with supporting different implementations via CompositionLocal or a global factory.
- as interop functions for external users, when they know that the current implementation is the AWT one.
In other words, we shouldn't use nativeClipEntry and derivatives (awtTransferable) in Compose itself, we should provide them for external interop.
The overall review and necessary changes to the current API will be done in https://youtrack.jetbrains.com/issue/CMP-7558/New-Clipboard-Adoption.-Review-API. For this PR, we should try to not make things more coupled.
| return readText()?.let { AnnotatedString(it) } | ||
| } | ||
|
|
||
| val transferable = nativeClipEntry as? Transferable |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Having readAnnotatedString isn't fine because of the issue I described, but because we already have it coupled in readText, it is fine for this PR, and it will be fixed together with the other functions later
45a28c9 to
7d6d0ec
Compare
Prior to this, the implementation of ClipboardUtils.desktop.kt would "flatten" AnnotatedStrings into their plaintext representation. This means that if you implemented a custom Clipboard, it would only receive the plaintext representation of the copied text in setEntry. With this change, the AnnotatedString is actually preserved when copying, so Clipboard can access the rich text and serialize it as it prefers.
7d6d0ec to
ad7f146
Compare

Prior to this, the implementation of
ClipboardUtils.desktop.ktwould "flatten"AnnotatedStrings into their plaintext representation. This means that if you implemented a customClipboard, it would only receive the plaintext representation of the copied text insetEntry. With this change, theAnnotatedStringis actually preserved when copying, soClipboardcan access the rich text and serialize it as it prefers.It should not change the observed behaviour when using the default
Clipboard, but allows users to create custom ones which can do things like JEWEL-815.PS: this removes the
withContextcalls fromClipboardUtils.desktop.ktsince they optimise for rare cases (extremely large content in the clipboard, probably O(100MB) or more) which are going to clog on the AWT side or during composition anyway. It can cause a small perf degradation in the overwhelmingly more common scenarios of tiny-to-large clipboard contents due to the dispatching of the coroutine to the IO dispatcher. This was discussed with someone from the Compose Text team.Testing
Ran tests, smoke tested by hand.
Release Notes
Features - Desktop
AnnotatedStringis available as a data flavor inClipEntry, instead of only its text.