Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

snakeyaml errors are not being reported to users #457

Open
mr-c opened this issue Sep 22, 2022 · 7 comments
Open

snakeyaml errors are not being reported to users #457

mr-c opened this issue Sep 22, 2022 · 7 comments

Comments

@mr-c
Copy link
Member

mr-c commented Sep 22, 2022

Is the disk full again @swzCuroverse?

Error: An unexpected error occurred when parsing the workflow

With https://github.com/emo-bon/pipeline-v5/blob/develop/workflows/gos_wf.cwl
@kinow
Copy link
Member

kinow commented Sep 22, 2022

I started the Docker Compose cluster locally using the latest code on master. Tried to import the workflow above, and got:

spring_1_905cde106a10 | 2022-09-22 23:10:49,101 WARN  [http-nio-8080-exec-10] org.commonwl.view.workflow.WorkflowController: url.parsingError WorkflowForm [url=https://github.com/emo-bon/pipeline-v5/blob/develop/workflows/gos_wf.cwl, branch=, path=]
spring_1_905cde106a10 | org.yaml.snakeyaml.parser.ParserException: while parsing a flow mapping
spring_1_905cde106a10 |  in 'reader', line 34, column 19:
spring_1_905cde106a10 |       ncrna_tab_file: {type: File?}
spring_1_905cde106a10 |                       ^
spring_1_905cde106a10 | expected ',' or '}', but got ?
spring_1_905cde106a10 |  in 'reader', line 34, column 30:
spring_1_905cde106a10 |       ncrna_tab_file: {type: File?}
spring_1_905cde106a10 |                                  ^
spring_1_905cde106a10 | 
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.parser.ParserImpl$ParseFlowMappingKey.produce(ParserImpl.java:905)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:171)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:57)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:43)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:136)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:116)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeScalarNode(Composer.java:221)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:190)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeValueNode(Composer.java:321)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeMappingChildren(Composer.java:312)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:293)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:194)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeValueNode(Composer.java:321)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeMappingChildren(Composer.java:312)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:293)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:194)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeValueNode(Composer.java:321)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeMappingChildren(Composer.java:312)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:293)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:194)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.getNode(Composer.java:116)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.getSingleNode(Composer.java:142)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:155)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:477)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.Yaml.load(Yaml.java:418)
spring_1_905cde106a10 | 	at org.commonwl.view.cwl.CWLService.yamlStreamToJson(CWLService.java:622)
spring_1_905cde106a10 | 	at org.commonwl.view.cwl.CWLService.parseWorkflowNative(CWLService.java:179)
spring_1_905cde106a10 | 	at org.commonwl.view.cwl.CWLService.parseWorkflowNative(CWLService.java:252)
spring_1_905cde106a10 | 	at org.commonwl.view.workflow.WorkflowService.createQueuedWorkflow(WorkflowService.java:349)
spring_1_905cde106a10 | 	at org.commonwl.view.workflow.WorkflowController.createWorkflow(WorkflowController.java:140)
spring_1_905cde106a10 | 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
spring_1_905cde106a10 | 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
spring_1_905cde106a10 | 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
spring_1_905cde106a10 | 	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
spring_1_905cde106a10 | 	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
spring_1_905cde106a10 | 	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
spring_1_905cde106a10 | 	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
spring_1_905cde106a10 | 	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
spring_1_905cde106a10 | 	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
spring_1_905cde106a10 | 	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
spring_1_905cde106a10 | 	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1067)
spring_1_905cde106a10 | 	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963)
spring_1_905cde106a10 | 	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
spring_1_905cde106a10 | 	at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
spring_1_905cde106a10 | 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:681)
spring_1_905cde106a10 | 	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
spring_1_905cde106a10 | 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
spring_1_905cde106a10 | 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227)
spring_1_905cde106a10 | 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
spring_1_905cde106a10 | 	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
spring_1_905cde106a10 | 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
spring_1_905cde106a10 | 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
spring_1_905cde106a10 | 	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
spring_1_905cde106a10 | 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
spring_1_905cde106a10 | 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
spring_1_905cde106a10 | 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
spring_1_905cde106a10 | 	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
spring_1_905cde106a10 | 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
spring_1_905cde106a10 | 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
spring_1_905cde106a10 | 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
spring_1_905cde106a10 | 	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
spring_1_905cde106a10 | 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
spring_1_905cde106a10 | 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
spring_1_905cde106a10 | 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
spring_1_905cde106a10 | 	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197)
spring_1_905cde106a10 | 	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
spring_1_905cde106a10 | 	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
spring_1_905cde106a10 | 	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
spring_1_905cde106a10 | 	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
spring_1_905cde106a10 | 	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
spring_1_905cde106a10 | 	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:360)
spring_1_905cde106a10 | 	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399)
spring_1_905cde106a10 | 	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
spring_1_905cde106a10 | 	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:890)
spring_1_905cde106a10 | 	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1743)
spring_1_905cde106a10 | 	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
spring_1_905cde106a10 | 	at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
spring_1_905cde106a10 | 	at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
spring_1_905cde106a10 | 	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
spring_1_905cde106a10 | 	at java.base/java.lang.Thread.run(Thread.java:833)

@kinow
Copy link
Member

kinow commented Sep 22, 2022 •
edited
Loading

Failed to parse on http://www.yamllint.com/ as well.

image

@kinow
Copy link
Member

kinow commented Sep 22, 2022

Looks to be an issue with the optional types, like File? inside the brackets. Rewriting obj {type: File?} to

obj:
  type: File?

everywhere in that workflow, and re-running YAMLint it passed validation. After some ~20 minutes I think, I got the graph for the workflow edited from my fork/branch:

https://view.commonwl.org/workflows/github.com/kinow/pipeline-v5/blob/eosc-life-gos/workflows/gos_wf.cwl

@kinow
Copy link
Member

kinow commented Sep 22, 2022

(might be something to add to the YAML topic in our user guide?)

@mr-c
Copy link
Member Author

mr-c commented Sep 23, 2022

Thanks @kinow for the investigation.

ruaml.yaml accepts this YAML, alas.

@mr-c mr-c changed the title server not accepting new workflows snakeyaml errors are not being reported to users Sep 23, 2022
@mr-c
Copy link
Member Author

mr-c commented Sep 23, 2022

Possibly the issue is that CWL is YAML 1.2, but snakeyaml is for YAML 1.1 ; maybe we need to switch to snakeyaml-engine or eo-yaml

https://bitbucket.org/snakeyaml/snakeyaml-engine/src/master/

https://github.com/decorators-squad/eo-yaml

@kinow
Copy link
Member

kinow commented Sep 23, 2022

Ah, that'd make sense. Shouldn't be too hard to test either of these libraries if they are published to Maven central repo (Java's PYPI equivalent I think).

mr-c added a commit that referenced this issue Sep 23, 2022
@mr-c
test for #457
c80a9d4
@mr-c mr-c mentioned this issue Sep 23, 2022
Merged
7 tasks
mr-c added a commit that referenced this issue Sep 24, 2022
@mr-c
test for #457
dc7c03d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kinow @mr-c