Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How could you add parallelism to make the encoding faster? #7

Open
smahm006 opened this issue Mar 13, 2023 · 6 comments
Open

How could you add parallelism to make the encoding faster? #7

smahm006 opened this issue Mar 13, 2023 · 6 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@smahm006
Copy link

On line 140-141 of lib.rs, there is a comment where the author mentions he tried threading with rayon but noticed it wasn't much faster than python threads.

Currently the python version gets me the token length in ~0.26 seconds while this crate takes ~1.8 seconds so I propose we should add back threading to speed up the process.

Now I am still a bit new to Rust so this post is more to bring suggestions on how would we go integrating threading?

@zurawiki
Copy link
Owner

Good points. How are you benchmarking these times today?

There is some overhead with the pyo3 code as it stands and I hope we can optimize that away once openai/tiktoken#40 and openai/tiktoken#50 land

@zurawiki zurawiki added the enhancement New feature or request label Mar 23, 2023
@zurawiki zurawiki added the help wanted Extra attention is needed label Apr 5, 2023
@oslfmt
Copy link

oslfmt commented Jul 18, 2023

I'd like to contribute to this issue. Is this project still active?

@zurawiki
Copy link
Owner

Issue is open and the project is active! :) Happy to advise / review any PRs

@oslfmt
Copy link

oslfmt commented Jul 20, 2023

Sweet! Besides this issue, any other notable issues/enhancements to work on? Gonna take a closer look tomorrow.

@zurawiki
Copy link
Owner

zurawiki commented Jul 20, 2023

Great to hear! Issues that can be worked on are listed here in the GitHub issues. I'd recommend tackling each issue one at a time. You can comment on each issue that you're interested in working on.

@zurawiki zurawiki added the bitbuilder:create Assigns BitBuilder to create a Pull Request for this issue. label Jul 25, 2023
@ellipsis-dev ellipsis-dev bot removed the bitbuilder:create Assigns BitBuilder to create a Pull Request for this issue. label Jul 25, 2023
@ellipsis-dev
Copy link

ellipsis-dev bot commented Jul 25, 2023

Sorry, BitBuilder couldn't generate a pull request for you. Please try again later. (wflow_xDf68BFfsE8dDc4G) 🤖

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

3 participants