Replace existing RealtimeBox implementation with RealtimeBoxBestEffort implementation #146

firesurfer · 2024-01-25T06:54:58Z

Hi as said in #139 this PR replaces the existing RealtimeBox implementation with the new implementation provided in #139 . I merged the tests into a single file and they work fine (even though the existing implementation had only a very small amount of tests)

The implementation of the get/set methods matches exactly the one from the existing implementation apart from some template magic which disables the standard get/set methods for pointer types. (For which they wouldnt provide any thread safety btw from my point of view - see #138)

TLDR: Merging this has the potential to break existing code if someone used pointer types with the RealtimeBox. For everything else it should work just fine.

I would also be happy to provide some documentation on how to use this implementation but I am not sure where in the repository it should go.

This PR should be merged after #139

EDIT: One question regarding the copyright notice for the existing tests. How should this be handled? At the moment I didn't copy any of it to the merged file.

…meBox, enable/disable certain methods for pointer types, Box is now usable for pointer types

…tests into a single file and adapted the naming of the existing tests for the RealtimeBox

codecov-commenter · 2024-01-31T18:20:31Z

Codecov Report

Attention: Patch coverage is 90.00000% with 4 lines in your changes missing coverage. Please review.

Project coverage is 75.36%. Comparing base (a11e9cf) to head (02df04a).

Files with missing lines	Patch %	Lines
include/realtime_tools/realtime_box.h	90.00%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #146   +/-   ##
=======================================
  Coverage   75.36%   75.36%           
=======================================
  Files           8        7    -1     
  Lines         341      341           
  Branches       62       62           
=======================================
  Hits          257      257           
  Misses         50       50           
  Partials       34       34

Flag	Coverage Δ
unittests	`75.36% <90.00%> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
include/realtime_tools/realtime_box.h	`90.00% <90.00%> (-10.00%)`	⬇️

firesurfer · 2024-02-01T07:09:29Z

@christophfroehlich I will update this PR as soon as #139 is finished

firesurfer · 2024-03-01T09:30:11Z

Fixes #138

Should fix pre-commit formatting warning

firesurfer · 2024-04-05T06:55:26Z

@christophfroehlich Given that #139 is now merged I updated this PR.
Please see my notes in the initial description about open points for discussion.

Also pinging @saikishor

firesurfer · 2024-04-05T07:12:34Z

@christophfroehlich Just a note about the CI. Interestingly I just had the case that pre-commit in the CI failed but did run sucessfully on my system. (Note about using the right include guard - REALTIME_TOOLS__REALTIME_BOX_H_)

firesurfer · 2024-04-05T08:01:47Z

Strange. Have you run pre-commit run --all? otherwise it checks only the changed files in the index.

Yes that's what I did run. But I couldn't reproduce it now. .

christophfroehlich · 2024-04-05T08:20:12Z

I would also be happy to provide some documentation on how to use this implementation but I am not sure where in the repository it should go.

We don't have a RST documentation of this repo, but I plan to rework the API documentation somewhen on control.ros.org. For now, please just add anything in the manner of rosdoc2 to be published on index.ros.org

saikishor · 2024-04-05T08:49:48Z

Is there a way to add some methods with old pointers and deprecation warnings?

saikishor · 2024-04-05T08:52:46Z

Is there a way to add some methods with old pointers and deprecation warnings?

if this is not possible, I don't think it is nice to completely override it and force everyone to change. If we want this to be the default implementation, then we would need to add a deprecation message to the old one and ask everyone to switch to the new one.

firesurfer · 2024-04-05T09:04:57Z

The current implementation in the RealTimeBoxBestEffort basically disables the normal trySet, tryGet...etc methods for pointer types.

Given that it is inherently unsafe to use pointers with this implemenation and the default trySet,tryGet methods (and also the existing RealTimeBox implementation) one could argue that it could be profitable to actually break builds here.

Nevertheless I completely understand your argument.

I guess adding a deprecation notice might be possible but I am not sure what is the best way at the moment. We could use SFINEA to provide a version of the class for pointers that include the deprecation notices (and removes the typename std::enable_if_t<!is_ptr_or_smart_ptr<U>, U> for the normal trySet, tryGet methods).

In the end I think this really comes to the question: Allow wrong behavior or possibly break builds?

saikishor · 2024-04-05T09:53:35Z

The current implementation in the RealTimeBoxBestEffort basically disables the normal trySet, tryGet...etc methods for pointer types.

Given that it is inherently unsafe to use pointers with this implemenation and the default trySet,tryGet methods (and also the existing RealTimeBox implementation) one could argue that it could be profitable to actually break builds here.

Nevertheless I completely understand your argument.

I guess adding a deprecation notice might be possible but I am not sure what is the best way at the moment. We could use SFINEA to provide a version of the class for pointers that include the deprecation notices (and removes the typename std::enable_if_t<!is_ptr_or_smart_ptr<U>, U> for the normal trySet, tryGet methods).

In the end I think this really comes to the question: Allow wrong behavior or possibly break builds?

I already agree with you on the unsafe part from the other PR, the problem here is we have a single branch that runs for Rolling, Iron, and also Humble, so breaking it would be breaking the setup of a lot of users, and such API breaking is usually not seen good.

I could think of the following 3 options:

Branch the rolling from others and then make this RealTimeBoxBestEffort as default over there.
Add a deprecation notice on the current one and ask users to move to the updated one this deprecation can be completely removed after one ROS release, so the users can migrate gradually.
We can somehow try to wrap the raw pointer of the user to a smart pointer and then use the current class.

What do you think about this?

firesurfer · 2024-04-05T11:51:30Z

Branch the rolling from others and then make this RealTimeBoxBestEffort as default over there.

This introduces additional maintenance effort -> I can not judge if this would be worth the effort

Add a deprecation notice on the current one and ask users to move to the updated one this deprecation can be completely removed after one ROS release, so the users can migrate gradually.

According to https://stackoverflow.com/questions/48045559/how-do-i-declare-sfinae-class I could do that.
The idea would be: Provide a specialisation for pointers, enable the methods there and add deprecation notes to them

I am currently trying to figure out a way to do this

We can somehow try to wrap the raw pointer of the user to a smart pointer and then use the current class.

This does not solve the problem. As the aim of this implementation is to provide thread safe access to its contents the access to any pointer type has to be wrapped in the way shown in the examples. (Using the shared_ptr does not make it safer than using a raw pointer)

firesurfer · 2024-04-05T11:55:21Z

@saikishor I just pushed a version which add a specialisation for pointer types. The tests will now show a deprecation note if get/set are used with pointer types. Also the code starts looking like a stl class...not sure if thats a good thing :D

christophfroehlich · 2024-04-05T18:47:16Z

EDIT: One question regarding the copyright notice for the existing tests. How should this be handled? At the moment I didn't copy any of it to the merged file.

Please just add another line to the copyright claim and don't remove the old one if old code remained inside (at least I saw this in other ROS repos)

firesurfer · 2024-04-15T06:09:06Z

@saikishor Perhaps you'll find some time for another review? It will now handle pointers types as before but print a deprecation warning

firesurfer · 2024-06-02T06:50:51Z

Just a friendly ping :)

bmagyar · 2024-07-03T17:27:17Z

include/realtime_tools/realtime_box.h

+   * @return false if the mutex could not be locked
+   * @note only safe way to access pointer type content (rw)
+   */
+  bool trySet(const std::function<void(T &)> & func)


Suggested change

bool trySet(const std::function<void(T &)> & func)

bool try_set(const std::function<void(T &)> & func)

we've been following snake-case in ros2_control, please adjust the rest too

To be honest I am a bit puzzled about this comment given that the PR that introduced these names was accepted #139

This PR ist just for replacing the old RealtimeBox with the new implementation while still keeping it backward compatible with the corresponding deprecation warnings.

Btw. @christophfroehlich As far as I know it is possible with clang-tidy to enforce such code styles. Perhaps you might want to look into that :)

saikishor

@firesurfer few minor comments.

Thank you and sorry for the huge delay

saikishor · 2024-07-17T14:03:32Z

include/realtime_tools/realtime_box.h

+template <class T, typename mutex_type>
+class RealtimeBox<T, mutex_type, true>


Suggested change

template <class T, typename mutex_type>

class RealtimeBox<T, mutex_type, true>

template <class T, typename mutex_type>

[[deprecated("RealtimeBox is not recommended to use with Raw pointers! Use smart pointers instead.")]]

class RealtimeBox<T, mutex_type, true>

It should actually be: Only use the "get/set(std::function) overloads of the RealtimeBox (which is the reason why we added this template specialisation for pointers).

But that deprecation message is already printed when the default get/set methods are used with pointers!

Isn't it wiser to mark the class as deprecated?, as you know from the beginning that type is not supported

Perhaps I should give a quick recap why it is designed like this given that quite some time has passed now:

The existing RealtimeBox implementation is not safe to use with any kind of pointer. Because only any operation on the pointer itself would be locked by the mutex but not the access to the data behind the pointer

The RealtimeBoxBestEffort solves this issue by providing additional overloads to get/set which take a function. These overloads lock the mutex and call the function. Inside this function manipulating the pointer is safe.
Additionally if the Type is a pointer the normal get/set methods which return a reference to the internal value are disabled via SFINEA (std::enable_if)

It makes sense to replace the existing RealtimeBox with the RealtimeBoxBestEffort (This is what this PR aims for). Additionally the requirement was made: We shall not break existing code, but we want to display deprecation notices if the default get/set methods are used with pointers. Therefore I added a template specialisation for pointers removes the std::enable_if for get/set with pointer types and adds deprecation notices for the get/set methods without the std::function overloads.

This should encourage users to migrate to the get/set(std::function) version for pointer types.
So in a future release the pointer specialisation of the class can be removed.

So to answer your question:

Pointer types are supported BUT you need to use the get/set(std::function) methods.

@firesurfer I understand your point. What I'm saying is, we don't have to break the Raw pointer in this version, but in the future versions, we can make the users use the smart pointers. So, if we mark the class deprecated with the notice to use the smart pointers in future releases, the users will be able to migrate in the meantime.

If we mark the methods alone, they still think the Raw pointers will be supported in the future. I'm not sure, if it is interesting to support Raw pointer in the future. If we still want to support, then the way you marked the depreciation is good enough @bmagyar @christophfroehlich any comments on this?

saikishor · 2024-07-17T14:07:03Z

include/realtime_tools/realtime_box.h

+ * the value behind the pointer
+*/
+template <class T, typename mutex_type>
+class RealtimeBox<T, mutex_type, true>


@firesurfer In this overlayed class, do we need to have a copy of most of the methods similar to the previous one? or is there a way to reuse them?

@saikishor From the "base" implementation:

// Provide a specialisation for non pointer types
// NOTE: When migrating to a safe access only version just remove the specialisation for pointer
// and let this be the only version!

My argument would be that in the longterm the pointer specialisation should be removed. I think the only save way to access the values behind pointers is to use the "get/set(std::function) overloads).
By keeping the code duplicated we can simplify this process.

My suggestion is to keep this version for at least one release and remove the pointer specialisation in the next one.

Hmm! @bmagyar @christophfroehlich any strong opinion on this one?

saikishor · 2024-07-17T14:08:11Z

test/realtime_box_tests.cpp

+  // This does not and should not compile!
+  // auto value = box.get();


Suggested change

// This does not and should not compile!

// auto value = box.get();

At the moment this would compile btw. (Because we added the pointer specialisation for compatibility reasons)

Sure, then let's remove the comment.

test/realtime_box_tests.cpp

Co-authored-by: Sai Kishor Kothakota <[email protected]>

Lennart Nachtigall and others added 16 commits January 14, 2024 10:33

restructured interface in oder to allow replacing the existing Realti…

b02e501

…meBox, enable/disable certain methods for pointer types, Box is now usable for pointer types

added some additional comments

613c55b

fix bug

93e1401

fixed wrong return type in templated method

fc2e4a2

added copyright notice

dce1458

Merge branch 'master' into firesurfer/master

63c51a7

Fixed issues due to merge with master

eaedafe

Merge branch 'master' into master

e666b93

Merge branch 'master' into master

aa62239

fixed spelling mistake

a22d150

Replaced existing RealtimeBox with RealtimeBoxBestEffort, merged the …

65bca5d

…tests into a single file and adapted the naming of the existing tests for the RealtimeBox

fixed ament_cpplint issues

186b4aa

fixed last remaining cpplint issue

a8b3f8a

suppress cppcheck missingReturn

69b224c

Merge branch 'master' into master

250fb85

Merge branch 'master' into replace_rt_box

e8372d6

try to resolve comments wherever possible

220111a

firesurfer and others added 6 commits March 1, 2024 10:32

Should fix pre-commit formatting warning

c2900a3

Should fix pre-commit formatting warning

Merge branch 'master' into master

6499036

Implemented suggestions

4fce184

Merge branch 'master' into master

d7a520c

Merge remote-tracking branch 'orig/master'

eaed073

merged main into branch

b679e9f

run precommit

ccd13a0

precommit should be satisfied now

9e0ce28

provide specialisation for pointer with deprecation notice

e3c0d8a

firesurfer mentioned this pull request Apr 5, 2024

Comment in realtime_box might be wrong and misleading #138

Closed

christophfroehlich marked this pull request as draft April 5, 2024 18:42

christophfroehlich marked this pull request as ready for review April 5, 2024 18:42

github-actions bot requested review from aprotyas, arne48, bijoua29, bmagyar, christophfroehlich, destogl and VX792 April 5, 2024 18:42

christophfroehlich requested a review from saikishor April 5, 2024 18:43

Adapt copyright notice

d3869d2

Merge branch 'master' into replace_rt_box

e251b00

bmagyar reviewed Jul 3, 2024

View reviewed changes

saikishor requested changes Jul 17, 2024

View reviewed changes

destogl and others added 3 commits October 9, 2024 19:12

Update test/realtime_box_tests.cpp

85a421e

Co-authored-by: Sai Kishor Kothakota <[email protected]>

Merge branch 'master' into replace_rt_box

f97e97c

Add missing include

02df04a

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace existing RealtimeBox implementation with RealtimeBoxBestEffort implementation #146

Replace existing RealtimeBox implementation with RealtimeBoxBestEffort implementation #146

firesurfer commented Jan 25, 2024 •

edited

Loading

codecov-commenter commented Jan 31, 2024 •

edited

Loading

firesurfer commented Feb 1, 2024

firesurfer commented Mar 1, 2024

firesurfer commented Apr 5, 2024

firesurfer commented Apr 5, 2024

firesurfer commented Apr 5, 2024

christophfroehlich commented Apr 5, 2024

saikishor commented Apr 5, 2024

saikishor commented Apr 5, 2024

firesurfer commented Apr 5, 2024 •

edited

Loading

saikishor commented Apr 5, 2024

firesurfer commented Apr 5, 2024

firesurfer commented Apr 5, 2024

christophfroehlich commented Apr 5, 2024

firesurfer commented Apr 15, 2024

firesurfer commented Jun 2, 2024

bmagyar Jul 3, 2024

firesurfer Jul 4, 2024 •

edited

Loading

saikishor left a comment

saikishor Jul 17, 2024

firesurfer Jul 17, 2024

saikishor Jul 17, 2024

firesurfer Jul 17, 2024

saikishor Jul 29, 2024

saikishor Jul 17, 2024

firesurfer Jul 17, 2024

saikishor Jul 17, 2024

saikishor Jul 17, 2024

firesurfer Jul 17, 2024

saikishor Jul 17, 2024

	bool trySet(const std::function<void(T &)> & func)
	bool try_set(const std::function<void(T &)> & func)

		template <class T, typename mutex_type>
		class RealtimeBox<T, mutex_type, true>

		// This does not and should not compile!
		// auto value = box.get();

Replace existing RealtimeBox implementation with RealtimeBoxBestEffort implementation #146

Are you sure you want to change the base?

Replace existing RealtimeBox implementation with RealtimeBoxBestEffort implementation #146

Conversation

firesurfer commented Jan 25, 2024 • edited Loading

codecov-commenter commented Jan 31, 2024 • edited Loading

Codecov Report

firesurfer commented Feb 1, 2024

firesurfer commented Mar 1, 2024

firesurfer commented Apr 5, 2024

firesurfer commented Apr 5, 2024

firesurfer commented Apr 5, 2024

christophfroehlich commented Apr 5, 2024

saikishor commented Apr 5, 2024

saikishor commented Apr 5, 2024

firesurfer commented Apr 5, 2024 • edited Loading

saikishor commented Apr 5, 2024

firesurfer commented Apr 5, 2024

firesurfer commented Apr 5, 2024

christophfroehlich commented Apr 5, 2024

firesurfer commented Apr 15, 2024

firesurfer commented Jun 2, 2024

Choose a reason for hiding this comment

firesurfer Jul 4, 2024 • edited Loading

Choose a reason for hiding this comment

saikishor left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

firesurfer commented Jan 25, 2024 •

edited

Loading

codecov-commenter commented Jan 31, 2024 •

edited

Loading

firesurfer commented Apr 5, 2024 •

edited

Loading

firesurfer Jul 4, 2024 •

edited

Loading