Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Transform `examples/js` to support modules #9562

Open
kdex opened this Issue Aug 22, 2016 · 139 comments

Comments

kdex commented Aug 22, 2016 edited

The main source code has recently been ported to ES2015 modules (see #9310); however, some parts are still relying on global namespace pollution and have thus become unusable.

Specifically the things in /examples/js/ haven't been transformed to support modules yet; this makes them currently unusable from within environments such as webpack or SystemJS.

Edit, 11-20-2016: They can be used, but it requires a lot of setup: SystemJS, webpack. If you need assistance with these: Sorry, but this is the wrong thread to announce it! 😄

@kdex kdex changed the title from Transform examples/js to support modules to Transform `examples/js` to support modules Aug 22, 2016

kdex commented Aug 22, 2016 edited

Side note: You could theoretically use these from within webpack by injecting the global THREE variable (and others, depending on what you're trying to load) using imports-loader, but the problem still remains for SystemJS and native module evaluation that will eventually land in browsers.

Owner

mrdoob commented Aug 23, 2016

I don't know how this can be fixed. We can't turn those files into ES6 Modules because, not only browsers don't support them yet, we want to support old-ish browsers too.

So, as far as I can see, the "hack" for Webpack and SystemJS is the price to pay for now?

andrevenancio commented Aug 23, 2016 edited

Either things like examples/js/postprocessing/ become a module of themselves or we need to come up with some kind of plugin logic.

@mrdoob what if github.com/threejs becomes a thing? You can create different repos inside of it which have a similar build system as the current threejs, which can be used via modules or people can just download the legacy code in the build folder. Again with the postprocessing in mind, it will have THREE has a dependency, and it would be both a module or you can use the /build/EffectComposer.js for legacy code?

Owner

mrdoob commented Aug 23, 2016

@mrdoob what if github.com/threejs becomes a thing?

I have a hard time maintaining one single repo already 😕

the idea was to give you and the maintainers more control over the releases, a bit like https://github.com/airbnb or https://github.com/facebook I'm happy to pick up some bits and bobs as I need them but you'll end up with github.com/randomuser/effectscomposer github.com/randomuser2/orbitcontrols :(

Contributor

GGAlanSmithee commented Aug 23, 2016 edited

Does it really make sense to make the examples be modules though? If I understand it correctly, the examples are consumers of three, not a part of it. They are there to provide you with example usages of three.js.

I think a more correct way to go about it would be to move as much as possible to the core of three.js and rely on tree shaking to eliminate dead code. (I understand this will be problematic for users that does not want to use a build pipe of their own though..)

andrevenancio commented Aug 23, 2016 edited

We're not saying we should have the examples as modules. We're saying that some of the files referenced in the examples folder aren't yet modularised, like OrbitControls.js or EffectsComposer.js or many others that are very often used in demos, prototypes and even in production. Those files, should be, as I was suggesting, modules of their own, outside of three.js

Contributor

satori99 commented Aug 23, 2016

I agree in regards to OrbitControls and the the other control classes. They are too useful to be just examples.

I wrote a small universal module loader, which supports various module systems and falls back to global namespace. It makes it all work, just by including the script. Should I do a demo with it for the examples?

Owner

mrdoob commented Aug 23, 2016

They are too useful to be just examples.

What does that mean? Do you mean that they are too commonly used?

Contributor

satori99 commented Aug 23, 2016

Yeah exactly. Especially OrbitControls. It might just be me, but I end up including that file a lot.

danrossi commented Aug 23, 2016 edited

I've ported OrbitControls over to a module, you can do something similar. My version has some changes when to dispatch start events on movement.

It can then include it within Three.Js main app file ie

export { OrbitControls } from '../three-vr-orbitcontrols/src/OrbitControls.js';

I've had to include other examples that are not modules yet, you concat them in with rollup in the rollup script like so

var fileList = [
    "../three.js/examples/js/effects/StereoEffect.js",
    "../three.js/examples/js/effects/AnaglyphEffect.js",
    "../three.js/examples/js/controls/VrControls.js",
    "../three.js/examples/js/controls/DeviceOrientationControls.js"
];

var out = fileList.map(function(filePath){
    return fs.readFileSync(filePath, 'utf-8');
});


var footer = out.join("\n");


export default {
    entry: 'Three.js',
    dest: 'build/three.js',
    moduleName: 'THREE',
    format: 'umd',
    indent: '\t',
    plugins: [
        glsl()
    ],

    outro: outro,
    footer: footer
};

https://github.com/danrossi/three-vr-orbitcontrols

I've done something similar for the VREffect but have refactored and cleaned it up also.

https://github.com/danrossi/three-vreffect

Perhaps we could start a small initiative to modularize these commonly used helpers from the examples. Clean them up and improve the quality (For example I rebuild the FPS controller to include keyboard controls, events, etc).

Owner

mrdoob commented Aug 23, 2016

What will happen to the people that don't use modules?

kdex commented Aug 23, 2016 edited

I find myself in the same position as @satori99, often needing a way to use Projector or CanvasRenderer, both being far from just an "example" with a good 1000 lines each.

@mrdoob People that don't use modules could just be pointed to versions of the project that had a pure ES5 codebase, so that they can pull the code via a <script> tag via global namespace pollution.

Essentially, the jQuery project did the same thing to gradually save themselves from supporting legacy browser versions (i.e. make a cut somewhere, support-wise).

Owner

mrdoob commented Aug 23, 2016 edited

@mrdoob People that don't use modules could just be pointed to versions of the project that had a pure ES5 codebase, so that they can pull the code via a <script> tag via global namespace pollution.

So... maintaining two versions of the same code base?

Essentially, the jQuery project did the same thing to gradually save themselves from supporting legacy browser versions (i.e. make a cut somewhere, support-wise).

That's different. One thing is stopping supporting legacy browsers. What you guys are suggesting is stopping supporting new programmers.

Contributor

GGAlanSmithee commented Aug 23, 2016 edited

Before we continue this discussion, I think it is important to highlight that this is "only" affecting people that are using a bundler already (emphasis mine):

Specifically the things in /examples/js/ haven't been transformed to support modules yet; this makes them currently unusable from within environments such as webpack or SystemJS.

Users that just want to include examples in a page can still do so (correct me if I'm wrong)

<script type="text/javascript" src="three.js"></script>
<script type="text/javascript" src="examples/js/OrbitControls.js"></script>
<script type="text/javascript" src="myapp.js"></script>

The group of users that does use a bundler is ofcourse very large and not unsignificant (I am one of these), but given that these users are already using a bundler, it might not be crazy to require them to do some manual step if they want to include examples in a custom build of three.

For example, something in the lines of what @danrossi have done, but we could modify the ´npm build´ script to take an additional, optional, list of examples to include in the bundle.

My build is quite out there.

I have had to copy the rollup script config. the package json file into a seperate directory outside three.js. The Three.Js main app file which becomes out of sync with head commits of course and need to be updated manually.

the paths look like

project
   build-threejs
   three.js

I then reference the exports like this within the main app file. They have to be changed from the original reference path.

export { EventDispatcher } from '../three.js/src/core/EventDispatcher.js';

I've then included my refactored modules from examples.

The examples that have not been converted have to be appended after the build output with the example code above.

I've stripped the size by 200kb by removing exports that are not needed also from within this modified main app file.

Contributor

GGAlanSmithee commented Aug 23, 2016

Yeah, I guess concatenating files like that is a bit hacky and also negates some of the possible benefits..

@kdex, is your proposal to turn each example into an umd bundle? That would make it work in all env. at the cost of added noise..

kdex commented Aug 23, 2016 edited

So... maintaining two versions of the same code base?

My intent was to eventually kill off the way that three.js ships bundles that mess with global namespace. It might be too early for that, given that <script type="module"> and the System global are not quite there yet, but I don't think that any project should promote this way of shipping code to the user in the future anymore. Hence, what I meant to say was that three.js should eventually just work like so:

<!--
 Use the bundle if you need this to work with legacy browsers.
 This will inject `window.THREE`.

 WARNING: 0.80.2 is the last version that comes with `window.THREE`.
 If you need a newer version, please consider migrating to using modules.
-->
<script src="three.js"></script>

So that in the future, users could migrate to something along the lines of:

<script>
    /* This would be the version that supports `examples/js` to be consumed */
    import THREE from "./vendor/three/three.min.js";
    import CanvasRenderer from "./vendor/three/examples/js/renderers/CanvasRenderer.js";
</script>

Another (maybe cleaner?) approach worth discussing might be to move examples/js into the core.
Pros:

  • No breaking change in terms of deployment (see outlined approach above)
  • Three can be consumed from bundlers and browsers
  • Full support for import syntax without hacking globals in with something like imports-loader

Cons:

  • For people using <script> tags only, the consumed bundle size will increase by a lot

If there's more cons, feel free to bring them up; but as for the bundle size: This only concerns users that want to inject three into global namespace anyway. If they see that this hurts site performance, it will as well act as an incentive to migrate to a build environment with something like rollup or tree shaking and create personalized bundles themselves.

@GGAlanSmithee brings up another good idea: With UMD, the bundle size won't increase for legacy environments, it allows being used with import, allows the examples to be transformed to use ES2015+ and prevents them to be moved to the core. On the other hand, this also makes three.js's build pipeline a little more complicated.

Owner

mrdoob commented Aug 23, 2016

So that in the future, users could migrate to something along the lines of:

<script>
    /* This would be the version that supports `examples/js` to be consumed */
    import THREE from "./vendor/three/three.min.js";
    import CanvasRenderer from "./vendor/three/examples/js/renderers/CanvasRenderer.js";
</script>

That was super helpful to see! Thanks! Can't wait for browsers to start supporting modules.

mrdoob commented 7 hours ago
What will happen to the people that don't use modules?

they will still be able to use script tags in their html with references to either a CDN or a local copy like so:

<html>
    <head>
        <title>Three.js</title>
    </head>
    <body>
        <script src="https://cdn.rawgit.com/mrdoob/three.js/dev/build/three.js"></script>
        <script src="https://raw.githubusercontent.com/mrdoob/three.js/dev/examples/js/controls/OrbitControls.js"></script>
    </body>
</html>

One of the advantages, for me, of having three.js as an npm module, means I avoid having to add a script tag per library in the html, or merge vendor scripts, copy files around from src to dest folders etc.

But having three.js as a module but not being able to import in the same way, OrbitControls, PostProcessing, ExploderGeometry etc, its not convenient.

We can easily have the community helping exporting every utility on the examples folder to a modularised version with backward compatibility (as three.js currently is). But I reckon its something you should do (at least create the official repositories, so people can contribute to them), or we'll end up having a couple of different versions of OrbitControls like we currently do in npmjs.

kdex commented Aug 24, 2016 edited

@andrevenancio If every example becomes its own official npm module written with ES2015 modules, they would essentially all do import THREE from "three"; in order to provide functionality by using what this THREE instance provides it, right? This is both useful as well as dangerous.

If some guy's personal three.js project does an import THREE from "three"; too, there is a chance that the versions of his project's THREE instance and, say his three-canvas-renderer module's THREE instance are different and not thus not only prevent a proper rollup, but can also introduce weird bugs that are hard to debug (I ran into failing subclass checks before, having introduced two vastly different three.js files by accident via <script>. I'd imagine that the same could happen by dividing the examples into npm modules).

Wouldn't it be safer if we could have all that stuff in one place, e.g. something like

import THREE, { CanvasRenderer } from "three";

?

This would guarantee that the THREE instance for CanvasRenderer also matches the instance of THREE you're importing, as they both originate from the same npm module. This ensures that the rollup can be fully leveraged and should prevent weird bugs.

At least, that's the npm side of it. On the GitHub side, it's really a matter of personal preference if you want to split the examples up into different repositories.

Again, I never said examples should be their own npm module.
What I said, and keep trying to explain, is that some utilities that only can be found in examples/js/ should be moved elsewhere.

Example, OrbitControls, or everything in the postprocessing folder.

And yes, I agree with you, we should be able to import it such as

import THREE, { OrbitControls } from 'three'

The build could generate each file into place. into examples/js/ and can be imported as individual modules in src/examples/ or contrib/src/ ?

Contributor

borismus commented Aug 25, 2016

I'd also really like this to be possible, both via

require('./node_modules/three/examples/js/controls/VRControls'); 

and via modules:

import VRControls from './node_modules/three/examples/js/controls/VRControls'

Could either manually add JS boilerplate to the end of each example (doesn't scale):

if (module && module.exports) {
  module.exports = ModuleName;
} else {
  THREE.ModuleName = ModuleName;
}

Or do something more clever with a build step. Having all of the example code in a unified 'three' bundle seems not particularly friendly for non-npm users, unless I'm missing something.

Owner

mrdoob commented Sep 20, 2016

There is no way to do a ES6 module code that also works with non-ES6 code without transpiling, right?

Contributor

GGAlanSmithee commented Sep 20, 2016

@mrdoob the es6 modules format is still not implemented in any browser (or node) so it will require transpiling. Even when support is added to modern browsers there is the question of backwards compatability, so es6 modules will probably have to be transpiled for a long time

Contributor

satori99 commented Sep 20, 2016 edited

I did an experiment with all this new module stuff to see how small the result could be after tree shaking and minification. I included some example classes by copying the files and adding an export/import declarations manually. This was easy enough to do, and worked fine on the first try.

It is a hassle, but doesn't this come down to whether the examples/ files are just examples, or have some of them become too useful to be considered as just example code? I'm thinking of the post-processing and controls code mostly.

@GGAlanSmithee chrome and firefox both run es6 natively. It was a surprise. If you try and change the rollup mode from "umd" to "es" . It should still run. I do believe it has to be transpiled for a very long time sadly unless two versions are offered. At least the rollup transpiling is very small compared to just using babel.

Contributor

GGAlanSmithee commented Sep 21, 2016

@danrossi oh I did not know, thanks for telling me, I thought it was still some ways off. I agree that transpiling will still be required for a long time.

@trippedout trippedout referenced this issue Sep 21, 2016

Merged

Modules #9310

Cecil mentioned me but the message isn't here strange. You can't choose es for three.js because it's not entirely pure es6 yet. It's just using es6 module imports that rollup is converting.

Once you do, you can run babel through rollup.

Back to the examples. I see no reason the sources can't be in es6 format. And then transpiled into place. The files in the target directory examples/js is the transpiled files. The example sources could go into contrib/js ?

This means they could be imported into an es6 module project.

import Module from 'three.js/contrib/js/Module';
Contributor

cecilemuller commented Sep 23, 2016 edited

We did have a config for building an es bundle (it's just the glsl transform that prevents using src directly), the eventual issue was merely the existence of a second bundle might be too confusing for users apparently :/

Contributor

zhanghaowx commented Oct 9, 2016 edited

Thanks to @kdex for pointing out import-loader could be a workaround. It worked perfectly for me:

import "imports?THREE=three!loaders/OBJLoader";
Owner

mrdoob commented Oct 9, 2016

Thanks to @kdex for pointing out import-loader could be a workaround. It worked perfectly for me:

import "imports?THREE=three!loaders/OBJLoader";

Oh? How does that work?

kdex commented Oct 9, 2016

@mrdoob It's a Webpack thing. 😄 I mentioned import-loader. The plugin allows you to inject a global variable into your module's scope. So the global (!) namespace of your execution context still remains completely clean, but during "compilation", Webpack will be able to figure out where to lookup the binding for a module that only comes with a global variable.

Basically, this plugin currently acts as the bridge between projects that only offer you a global variable and proper ES2015 modules, at least in the webpack world.

The syntax that you can see above is weird. It roughtly translates to "Hey, imports loader! In this module, please assume that there's a global variable named THREE, and its value is whatever the npm package three exports. Also make sure to import three/loaders/OBJLoader.js, but don't bind it to any variable." (which should exploit the fact that it modifies the THREE object, so there's no need for a variable binding).

Beware: All of this only covers webpack. AFAIR, jspm currently doesn't come with such a plugin, and once browsers handle ES2015 modules natively, we're back to square one.

kdex commented Oct 9, 2016 edited

By the way: Earlier in this thread, I said that moving everything into the core might be safer, since it allows importing components as:

import THREE, { CanvasRenderer } from "three";

After some reflection (and comparison to what big libraries tend to do), I'm not too sure about that anymore. I'm not sure how efficient the best tree-shaking algorithm can be, but compare:

import THREE from "three";
import CanvasRenderer from "three/CanvasRenderer";

The difference is that by importing two exports from "three", your browser still needs to know the entire three module to know what the exports are. Now imagine what happens to the file size if we cram every export in there. 😕

I've seen big libraries like material-ui tackle this problem as shown above, too, and they tried the "single-module" way once, too.

Good news. you don't have to turn the entire library into Es6 to turn the modules into Es6 and pass through babel. This is one module turned into an Es6 module and explains the extra rollup configs.

If at any stage it does turn into Es6 code, eslint is a good code checker and helps check for code problems.

https://github.com/danrossi/three-vr-pointerlockcontrols

It's not very hard to do hybrid module/ global file actually. Will try to wrap my FPS controls into a forward/backward compatible file.

@killroy42 if you mean outside of the main build that is possible. My requirements is to build them in. Which I then concat at the top of my Es6 project.

My guess the main app for the module would just have something like

export { PointerLockControls } from './src/PointerLockControls';

I already do that in my projects, although using a loader stub that manages it. But I think you can create a hybrid .js file that can be used in either environment.

Contributor

delcore92 commented Oct 17, 2016 edited

I know how mrdoob feels about the accessibility of the project to newcomers. So hopefully a newbie's perspective helps here.

There are so many useful scripts nested within the project that I don't think belong directly under the THREE namespace, but rather a complementary one (or more). I feel that it would be beneficial to break these things out into seperate modules and house them under an organisation so that they may be maintained separately by the community. It will will take the load off the core repo and seperate concerns in regards to Documentation, PRs, and issues.

For example:

  • editor
  • examples
  • utils
  • threejs.org
  • etc.

For me as a newbie, the problem wasn't that everything was hard to get started, it was more that i had to delve into the codebase and try to figure out if a script was there and right for my needs. Separating the repos would also give contributors a chance to write some readmes and tests, whilst allowing three to scale.

Owner

mrdoob commented Oct 17, 2016

For me as a newbie, the problem wasn't that everything was hard to get started, it was more that i had to delve into the codebase and try to figure out if a script was there and right for my needs.

Not sure we're talking about the same type of newbie... A newbie for me is someone that wants to do a website without modules and/or build processes. Just a html and a bunch of js files.

Separating the repos would also give contributors a chance to write some readmes and tests, whilst allowing three to scale.

I don't think you're taking in consideration the amount of work that this would bring me...

completely understand where you're coming from, but why change three.js to a modular approach if we can't use all the plugins that depend on the global THREE variable? :( (well we can use them if we add a script tag to the html, but then we have different logics of importing files that complement themselves)

Owner

mrdoob commented Oct 21, 2016

completely understand where you're coming from, but why change three.js to a modular approach if we can't use all the plugins that depend on the global THREE variable? :(

Are you suggesting that we should revert the modules change? ;P

YES for my own benefit and NO because I'm afraid of the community going after me and hunt me down! :)

@kdex I'm current working on a Angular2+three.js app in TypeScript using angular-cli(the build system is webpack). I'm trying to load a model with the MTLLoader. After I installed the imports-loader and imported the three and MTLLoader as the way that you have suggested:

import * as THREE from 'three';
import "imports?THREE=three!../../node_modules/three/examples/js/loaders/MTLLoader.js";

When I do:

 var mtlLoader = new THREE.MTLLoader();

It says:

Property 'MTLLoader' does not exist on type 'typeof THREE'.

Looks like MTLLoader still not be able load into the THREE variable. Any Idea?

kdex commented Nov 13, 2016

@table315 You're not the first person to come to this thread looking for a way to import examples in a non-ES5 environment. 😕 Let's just make one thing clear for future readers:

This is not a general support thread. This thread is merely a discussion about the project's build pipeline and how we should enhance it for the future. If you need assistance with imports-loader, you can always look for examples or open an issue here.

Other than that, you might want to check everything after THREE=three!. This string looks wrong and differs a lot from what I've showed above. Lastly, please note that you can use GitHub to search for code examples, see here.

Near the beginning of this issue, there was mention of a hack for system.js that will let it import OrbitControls.js (systemjs can import old fashioned global modules) such that it "sees" THREE, thus install itself inside the THREE namespace.

What's the trick?

I use import * as THREE from 'three'; with no problem .. shouldn't I be able to load OrbitControls within the THREE scope? The problem is, naturally:

THREE.OrbitControls = function ( object, domElement ) {...}

.. so I guess I should be able to somehow make THREE visible to an import of OrbitControls, right?

kdex commented Nov 13, 2016

@backspaces Was there really? I think SystemJS uses System.config for setting up globals as dependencies in modules. If it helps, here's an issue on how to use three.js with SystemJS.

Unfortunately, it currently doesn't get any nicer than this, and making it work will require additional configuration. Hence this thread. 😄

backspaces commented Nov 14, 2016 edited

I've had good luck using System.js with commonjs and global modules. The main issue here is the direct install of the controls into THREE. My guess is that if the controls first built their module object in the global space, then installed it in THREE in the current namespace, it might work, and not break prior code.

I plan to grab the controls, and just run a simple script over them to install themselves in the global space first, then install them in THREE. Some fussing around is bound to work. And no, a script tag isn't OK. The locality of Modules is great, the import is where its needed. I'll bet a large percent of html files with over 20 script tags have scripts that are no longer used!

killroy42 commented Nov 14, 2016 edited

I use this module-loader.js
With this module loader it can work like this:
index.html:

<script src="/vendors/THREE/three_r83dev.js"></script>
<script src="https://gist.githubusercontent.com/killroy42/b8cbbd90b1209e8d8dec817ef88e8730/raw/dea5a7db1a26349b73e14f7f3ff5bca7c5408662/module-loader.js"></script>
<script src="/vendors/THREE/controls/OrbitControls.js"></script>
<script src="script.js"></script>

script.js:

const {
  OrbitControls,
  Box3, BoxGeometry, MeshBasicMaterial, SphereGeometry
} = require('THREE');

That's what I use...

backspaces commented Nov 16, 2016 edited

@mrdoob: This is a serious problem for those of us who have gone to es6 and modules. More serious than a simple "issue". It's the tipping point for Three: how to be "forward compatible"!

Modules are the most important problem because they completely change the way projects encapsulate their code. You have brilliantly gone to modules internally, Yay!, but have difficulties with the controls due to their directly installing into Three. This is fine, IMHO, it is Javascript after all!

There really are two problems:

1 - Three itself. I would prefer direct access to Three modules so that the module loader gives me only what I need. This, with an http/2 CDN, would likely improve Three performance.

2 - Legacy. AFAIK only controls are the problem? If so, I believe the problem can be solved by simply removing the direct installation into Three and letting them be what Guy Bedford (system.js) calls "global" modules. .. i.e. modules that simply install themselves in window. Those system.js can handle, witness the ability to import Three.

If a simple modification of controls can both remain backward compatible but load as an import, that's cool.

But face it: like all projects, Three has to cope with both backwards and forwards compatibility.
Suggestion: Talk to Guy to see what other's have had success with, OK? He's very approachable.

Owner

mrdoob commented Nov 16, 2016

Suggestion: Talk to Guy to see what other's have had success with, OK? He's very approachable.

Maybe you can do that for us?

@kdex I think you're overly optimistic about how soon that support will land, how quickly folks will update, and so on.

It's not nice to thrust build pipelines upon people just because you think it's the future.

kdex commented Mar 3, 2017 edited

@crertel-packlane Not really: Safari 10.1 already shipped them. Edge 15 already has them behind flags. v8 has modules behind flags, too, so Chrome, Opera and Chrome for Android should see support soon-ish.

What you said about build pipelines is beside the point: I didn't say that I'd like to force them upon developers for the future (!), I just said that supporting legacy environments (think of IE5) is nothing that any sane person should do without some form of automatic process like transpilation, not right now and not in the future.

Why not remove the examples from the three.js main repo completely? have OrbitControls (and others) as optional imports for the those using ES6 modules and let the tree shaking remove them if not in use, and the legacy three.min.js file with the window.THREE reference has all the utils in it. What's the worst that can happen? Having a couple of extra kb's to the file size?

Owner

mrdoob commented Mar 6, 2017

Why not remove the examples from the three.js main repo completely?

Having the examples in the same repo keeps me sane 😅

Three is an ecology. Yes, somewhat messy at times but always converging onto what we all want. I really am just fine with how it is with examples etc.

I even use the dat.gui and stats.js that come wrapped in the gift! What a deal.

I do look forward to a Three version 2 where lots of possibilities like webgl2, instancing, maybe the new TypedArray SIMD apis, and heck, web assembly too. But so far, so good.

halvves commented Apr 5, 2017

I've read this thread up and down and have seen a lot of good points for and against refactoring the examples section into modules. In the meantime I really needed/wanted VRControls and VREffect as easily importable modules in a few threejs/webpack projects I've been working on. There are other ways of getting these modules into a webpack build (some are discussed here), but they felt clunky at best to me. Until VRControls/VREffect show up as modules in either THREE or a separate package, these could be a useful resource for those of you trying to bundle modular VR projects:

halvves/three-vrcontrols-module
halvves/three-vreffect-module

I'll do my best to keep these abreast the current versions in THREE proper.

hccampos commented Apr 5, 2017

Really sounds like Three should just split things into separate packages (with multiple builds, as needed) and use something like https://github.com/lerna/lerna to keep things sane in terms of publishing/maintenance.

kdex commented May 9, 2017 edited

Native support for import/export and <script type="module"> has recently landed in Chrome Canary. Edge and Firefox already have support behind flags, and Safari even shipped with it in March. Looks like we're about to get into a state where we could work on a viable solution.

foobarbecue added a commit to foobarbecue/climbshare that referenced this issue May 17, 2017

Update Meteor release. This broke imports. Use require instead.
Hopefully three.js will be fully modular soon -- see
mrdoob/three.js#9562 . And hopefully meteor
will stop changing their loading methods all the time.

ChisholmKyle commented May 24, 2017 edited

I came across a method of wrapping OrbitControls.js (but works for anything using THREE as a global container). I use it with babel and browserify. First, wrap the source code as follows:

    module.exports = function (THREE) { 
        /* contents of OrbitControls.js */
    };

then from your source file, simply call OrbitControls(THREE) after importing:

    import * as THREE from 'three';
    import OrbitControls from '/path/to/OrbitControls';
    OrbitControls(THREE);

and now it can be used the exact same way you would use it in the examples:

    let controls = new THREE.OrbitControls(camera, canvas);

I set up a simple script (mac and linux) to wrap the files, and it works for the few loaders and controls I've tried:

    sed -i  "1s/^/module.exports = function (THREE) { /" "${src}"
    echo "};" >>  "${src}"

Note that mac needs to call sed with sed -i "" -e instead of sed -i.
Do you think it would be worth while to see if wrapping everything in examples/js like this would work?

amelierosser commented May 24, 2017 edited

@ChisholmKyle See https://github.com/mattdesl/three-orbit-controls/blob/master/index.js

Many have been using this approach for a while, far from ideal though.

ChisholmKyle commented May 24, 2017 edited by mrdoob

@amelierosser Ah yes! That's where I saw this method. I was thinking why not just modify the versions in mrdoob's repository (are they the most up-to-date and compatible?) for any or all example/js/* modules?

@ChisholmKyle Totally depends on how you use three in your project along with the examples js. I uses to use the approach you showed. Now I favor converting the examples js into actual es6 modules with this util:

https://gist.github.com/amelierosser/bb3bf98d934ca2d26774788a3f9eb803

Neat script! Thanks :)

@backspaces Yeah i could, although it doesn't work for all examples as some are wrote slightly differently. It would need a more robust script.

kus commented Jun 8, 2017

I came across this like many other people trying to import files from the three module examples folder in node_modules with ES6, babel, Webpack 2.

I noticed some people have created seperate NPM modules with individual files from the examples folder which I don't agree with. If there is an update they can fall behind, and why include another package when we already have all of them sitting there in node_modules.

In your webpack config add an alias:

resolve: {
	alias: {
		'three-extras': path.resolve(__dirname, 'node_modules/three/examples/js/')
	}
}

Install Webpack plugins npm install --save-dev imports-loader exports-loader

Import them with three-extras/ where you want to use them. Some of the files require THREE so you can inject it, and if the module doesn't export properly (OrbitControls) you can export it. Examples I have tested:

  • Detector import Detector from 'three-extras/Detector'
  • dat.gui import dat from 'three-extras/libs/dat.gui.min.js'
  • stats import Stats from 'three-extras/libs/stats.min.js'
  • OrbitControls import OrbitControls from 'imports-loader?THREE=three!exports-loader?THREE.OrbitControls!three-extras/controls/OrbitControls' // eslint-disable-line import/no-webpack-loader-syntax

DO NOT USE WEBPACK! MINIMIZE WORKFLOW! IT'S FINE THE WAY IT IS!

All transforms one needs are easily accomplished by Rollup, converting from es to cjs, iife, es (three.module.js .. i.e. make a larger modules out of several smaller ones)

Three code is a delight to read. Don't screw it up with the continuing pile of workflow, creating unreadable JS files.

Whew.

There is a simple solution to the examples/ problem. Simply ship the a node converter that creates a module from the existing THREE.foo = function .. format. I'm doing that as are several others.

And yes, the module <> legacy strategy is hugely difficult and varies between projects. We'll all have scripts making conversions here and there. dat.gui, stats.js (already es6 module), examples/* are all transformable to modules.

(aside re legacy dependencies, not THREE) Actually, a horrid but usable hack is to simply wrap a module around project's foo.min.js, with a mod or two for the horrible browserify hack they use. Works fine.

Rollup has a commonJS plugin for THREE legacy dependencies. dat.gui is even half way to es6 modules too, only having one silly module that imports a piece of raw html for a "save dialog". Make a pull request to fix it. All other problems are solved already.

Contributor

cecilemuller commented Jun 8, 2017 edited

@backspaces Perhaps there was a misunderstanding: @kus was referring to webpack loaders/alias in the end user's config (applications that are not only js libraries, but also with models, assets, textures or even extra shaders, hence Webpack is a good fit as it handles bundling and assets).

It wouldn't change the way Three itself is bundled by Rollup, so you don't have anything to worry about.

kus commented Jun 9, 2017

Sorry for the confusion @backspaces, as @cecilemuller pointed out I was purely talking about using three in your own project and getting access to the modules in the examples folder in a smart way with Webpack 2. The only reason I posted it as this page is one of the top results when I was searching for the solution, so I thought I would share it for any other people that land on this page looking for the solution.

Blush! That'll teach me to be a bit more discerning! It is interesting that there is an another clever way to manage the examples/, thank you. Strategies for module use are just now blooming and it's nice to have another.

My hair trigger was brought about by my converting to es6 Modules, thus looking at my dependencies and seeing whether or not they too were modules, thus "importable" rather than <script>able.

But I was dismayed to find I couldn't read several of them, due the "compiling". Several were close to being module based too.

A great example is the code at the top of the page generated by I think browserify. Sure was mystical! I finally understood it by pretty-ifying it. Fascinating! But kinda sad too.

Javascript was meant to be Write And Run, and we're only now getting back to it. Yay!

below webpack config works for me:

  resolve: {
    extensions: ['.js'],
    alias: {
      'three-OBJLoader': 'three/examples/js/loaders/OBJLoader.js'
    }
  },
  plugins: [
    new webpack.ProvidePlugin({
      THREE: 'three'
    })
  ]
Contributor

cecilemuller commented Jun 27, 2017 edited

Ideally, it would be convenient to use scoped NPM packages (right now the three username doesn't seem registered yet) to publish those extras as their own package.

Beyond making it easier to make use of those almost-core classes, it also opens the possibility to endorse thirdparty packages that don't have to be stored and maintained in the three.js monorepo yet can be in the @three namespace, and the ability to provide checkJs intellisense also for "examples", not just core.

If each class had its own package:

import NRRDLoader from '@three/NRRDLoader';

Or all examples in a single package:

import {NRRDLoader} from '@three/examples';

backspaces commented Jun 27, 2017 edited

If someone is interested, see how to integrate Three.js with examples/js into Angular typescript:
angular-three-examples

See demo including ColladaLoader and OrbitControls

Briefly:

  1. In tsconfig.json add "allowJs": true
  2. Copy required script from example/js into Angular application
  3. Add on top of example: THREE=require('three');
  4. Import it into component

What is the status of this ?

Is there a way to only make a build with what is needed ? I have managed to strip down all includes to a few exports and now the build is 360k.

It will still be including all shader programs not needed. I am using a custom one that isn't part of the sources.

tiborsaas commented Aug 27, 2017 edited

Hello,

I've created a Rollup plugin to load files from the examples folder:

https://www.npmjs.com/package/rollup-plugin-threejs-legacy-import

It automatically removes the THREE object namespace, generates exports and import statements.

Not perfect yet, but current version is good enough to test this concept

jokester commented Aug 30, 2017 edited

When developing a frontend app with webpack and TypeScript, I was able to bundle examples/js/controls/OrbitControls with only ts-loader and no other loaders. Guess I can share my configuration here for similar cases. (a minimal example can be found here).

package.json:

"dependencies": {
    "three": "^0.85.2",
    "@types/three": "^0.84.12",
    "ts-loader": "^2.1.0",
    "typescript": "^2.3.4",
    "webpack": "^2.6.1"
},

entrypoint.ts:

import * as THREE from "three";

// OrbitControls.js expects a global THREE object
(window as any).THREE = THREE;

// NOTE: OrbitControls must be included with require:
// using "import" cause it to be executed before global THREE becomes available
require("three/examples/js/controls/OrbitControls");

// ... code that uses THREE and THREE.OrbitControls

NOTE: webpack may warn like export 'OrbitControls' (imported as 'THREE') was not found in 'three'. I suppose we can just ignore this warning.

dyh333 commented Sep 20, 2017

@makimenko I am trying the similar thing using ng4 + threejs. your repo is work well but can it import threex such as threex.domevents(https://www.npmjs.com/package/threex.domevents)?
thanks.

@dyh333 recommendation:
If you want to select and pick scene object use standard THREE Raycaster - see NG4+Three.js example.
If you want to listen host events (key press, mouse etc.) use standard ng4 @HostListener annotation

DieAlchemistenAG commented Oct 2, 2017 edited

I had this Problem, when I was porting an existing Project to a newer version. I had to use webpack, because my UI is made with React. On top of that, I was using lots of code from the examples. Here is how i fixed the problem for me:

The code below shows how I import everything I need and Inject the THREE namespace into the imported files, after this, you just import what you need from 'my-three'.
Note the loaders section of the webpack.config, some of the extensions need GLSL files

1): Fix add aliases in webpack config (webpack.config.js)

var path = require("path");
var ClosureCompilerPlugin = require('webpack-closure-compiler');

module.exports = {
    //...
    resolve: {
        //...
        alias: {
            'three-ext': 'three/examples/js',
            'my-three': path.resolve(__dirname, './src/js/three/three-namespace.js')
        }
    },
    loaders: [
        // three js imports glsl, provide a loader for that!
        {
            test: /\.glsl$/,
            loader: "raw"
        },
        // if something is imported via "my-three" the THREE context will also be available
        {
            test: /[\/\\]node_modules[\/\\]three[\/\\]examples/,
            loader: "imports?this=>window,THREE=my-three"
        }
    ],
    plugins: [
        // not needed, but makes compile mutch smaller
        new ClosureCompilerPlugin({
            compiler: {
                language_in: 'ECMASCRIPT6',
                language_out: 'ECMASCRIPT5',
                compilation_level: 'ADVANCED'
            },
            concurrency: 3
            // ... set your own options
        }),
    ]
}

2): Add the declaration (in src/js/three/three-namespace.js)

// Global namespace
import THREE from 'three';

// first just import what you need from examples
import "three-ext/controls/OrbitControls.js";

// Export
export let OrbitControls = THREE.OrbitControls;

// export THREE again, the imports should be mixed into the THREE namespace
export * from THREE;

3): Use threejs as you like

import { Scene } from 'my-three';

let scn = new Scene();

// or the "three" way
import * as THREE from 'my-three';

let scene = new THREE.Scene();

// should print true ;) 
console.log(typeof scn === typeof scene);
Contributor

trusktr commented Oct 3, 2017

Hello all, we can make pull requests. Here's one for CSS3DRenderer: #12320

Contributor

trusktr commented Oct 3, 2017

Simple idea: we can make multiple entry points if intent is really to keep the main global build as small as it is now. src/Three.js is the default entry point. There could be src/Three.full.js which imports everything. Etc. The build step can compile each entry point instead of just one entry point like it does now. Global polluters can choose which file to pollute with.

If you rollup your app along with the three source code, which is already es6 modules, you'll get just what you use. I think that is one of the reasons es6 modules exist!

Similarly, you can just create your own module that imports only what your code will use from three and rollup that along with the three source. That should also work.

This is one of the issues facing "modern" JS repos, however .. whether to include the individual modules. The files could easily "move" code around, and change, yet not change the high level API. Buyer Beware sort of thing.

Certainly would be interesting tho, to see how well it works!

Let me be clearer. My repo has a "legacy" bundle along with both the es6 source and an es6 rollup, just as three does. But one user asked to separate the Model from the View. OK, now four bundles and source.

Oh wait, they wanted multiple Views (Three, Canvas2D, and a NullView). You can see where this is going! This is a huge issues with modern repos and I think it hasn't been completely sorted among the pioneers. See these two articles:

https://medium.com/@backspaces/es6-modules-part-1-migration-strategy-a48de0b7f112
https://medium.com/@backspaces/es6-modules-part-2-libs-wrap-em-up-8715e116d690

Contributor

trusktr commented Oct 3, 2017 edited

Babel nowadays can convert any module into UMD-based global.

Let's look at popular libraries like React, Vue, and others. They have something in common today: they all have a module-first approach, and most people use that approach. They also have high numbers of adopters. The libs also offer globals for those who need to incrementally adopt the libs, with clear mention from the library authors that globals are not the recommended way to use these libs in production.

Three.js could do the same. It will not hurt Three's popularity.

Get easily started prototyping with Three.js by including it in an HTML page via script tag:

<script src="path/to/three.js"></script>

However note that this isn't ideal for production because the Three.js bundle is huge. By using a modern build system like ___, ___, or ___ you can import only the parts of Three that you use, and create lean bundles, which is common practice for production apps.

This would need to be updated to also mention, and show example of, native ES2015 modules when those become available (but I still recommend a bundler over native es2015 modules, for performance reasons because loading a single file is still faster than loading many modules separately, even faster than the early HTTP/2 servers that we have today. Too lazy to cite sources ATM).

@backspaces
"If you rollup your app along with the three source code, which is already es6 modules, you'll get just what you use. I think that is one of the reasons es6 modules exist!"

Sadly no, it still imports almost the whole codebase even when if I use tree shaking.

Collaborator

donmccurdy commented Oct 4, 2017

If the goal is to support ES6 modules in examples/js/*, then I don't think including everything by default — and assuming users will all have tree-shaking — is a preferable way to accomplish that.

One alternative would be to convert all examples to ES6 in-place.

  • Pros: Build is still small, examples can be used with ES6 modules. Some examples (loaders, postprocessing) might benefit from use of imports.
  • Cons: Users must transpile examples before using with older browsers. The three.js website examples might not work on older browsers, without some effort.

We could set up build scaffolding to automatically transpile each of the examples (individually) to build/examples/*. But this would require some maintenance to make sure the right things are exported, and makes npm run dev much more complex...

Contributor

looeee commented Oct 4, 2017

This argument keeps going round in circles 😆

Here's one other solution:

  • keep the examples almost exactly as they are (some may require minor refactoring)
  • provide a small node.js utility script that converts them to a simple ES6 module. ExampleName.js -> ExampleName.module.js

Let users run the script themselves for an examples they need to import.

In the case of (e.g.) OrbitControls, the script just needs to:

  • Add import * as THREE from 'three' to the top of the file
  • Replace THREE.OrbitControls = function ( object, domElement ) {
    with export default function OrbitControls ( object, domElement ) {
  • Change THREE.OrbitControls -> OrbitControls wherever it appears in the file.

Then to use OrbitControls, you can just do:

import OrbitControls from './OrbitControls.module.js

Contributor

Itee commented Oct 6, 2017

@looeee

I had a similar script but using gulp ( that could, during a transition time, allow to do exactly what we want... ).

But like @amelierosser commented on 24 May
Or @tiborsaas commented on 27 Aug

But just i little question...

The goal is: Do not break backward compatibility for older browser, which browser, in which version, should we support ie7 too ???

WHY DON'T BREAK BACKWARD COMPATIBILITY (like a major version) ??? To allow big clean up, and set three.js to full es6 modules support, and remake the folder tree ? Why is this so important ? All big library did things like this, no ? And you need to upgrade your database (oh damned threejs made a major release, i will need to change some script tags... uh uh uh... ) !

To avoid big library size ? Seriously ??? In 2018 we have 4G, ADSL+, Fiber, etc... but you can't load 5Mb more ??? Really ??? Especially since developpers bundle threejs in there app, use compression etc...
And ok right... you had load a little three.min.js (of 503ko) very fast with your 56kb modem... and then what...
You can't load real model or 3d application if you can't load a full threejs, no ??? So the library size is a totally wrong problem, because any 3d app will consume much more data than the full, not tree shaked, not compressed, not minimified, three.js library. A simple texture image ? BOOM 3Mo...

And finally, all people will be aware of this, no ? And all people that need previous version could still use too, no ?!?

So what is the problem to make a break ?

Contributor

looeee commented Oct 7, 2017

In 2018 we have 4G, ADSL+, Fiber, etc...

Maybe you live in a country with good internet, but no, everybody in the world does not have a 5mb+ connection in 2017, and they still won't in 2018...

Owner

mrdoob commented Oct 7, 2017

  • provide a small node.js utility script that converts them to a simple ES6 module. ExampleName.js -> ExampleName.module.js

Yeah, that would be ideal. But I was unsure whether to have 2 files for each. People wouldn't know which one to contribute to.

Contributor

trusktr commented Oct 7, 2017 edited

Yeah, that would be ideal.

That's more work than just moving them into src.

As @tristanvalcke mentioned, the size of Three.js minified and gzipped even with all example code moved into src (and not tree shaken), is small compared to the size of assets (textures, images, etc) for a decently awesome 3D experience.

Another idea: What about two entry points, like three.full.js and three.lite.js, where three.full.js imports the re-exports everything from three.lite.js plus the extras (and demos use the bundle generated from the three.full.js)? That would be really easy to implement (just duplicate the existing rollup step with alternate input file and alternate output file), without a whole new transpile step to maintain for generating possibly confusing module versions of the example files.

Collaborator

donmccurdy commented Oct 7, 2017 edited

As @tristanvalcke mentioned, the size of Three.js minified and gzipped even with all example code moved into src (and not tree shaken), is small compared to the size of assets (textures, images, etc) for a decently awesome 3D experience.

I disagree on this. Three.js is used for many things, like generative art and data visualization, which do not need any models or textures. It is a general 3D library, not a game engine, and developers targeting mobile devices may reasonably want their page to be well under ~1MB in size.

And besides, many things in examples/js don't belong in src/, or can't be moved. There are ~40 loaders in the examples/js/loaders directory (including DRACOLoader which has large emscripten and WASM dependencies) and there's just no reason to ship all of that as part of any three.js binary, even with a "full" label.

That said, if there are key things that are commonly used (like OrbitControls maybe?) I certainly have no problem with moving those into src/ on a case-by-case basis.


ES5 source → ES6 modules generated

  • provide a small node.js utility script that converts them to a simple ES6 module. ExampleName.js -> ExampleName.module.js

Yeah, that would be ideal. But I was unsure whether to have 2 files for each. People wouldn't know which one to contribute to.

Perhaps the generated version could go in another folder like build/examples/*, or even be minified, so it's obvious which to contribute to.


ES6 modules source → ES5 + global exports generated

In the long run, I'd rather be able to write ES6, and eventually ES7 async and await, and have the ES5 + THREE.Foo global export version be generated from that... in theory we could do this:

  1. manually refactor examples as ES6 modules (perhaps each still relying on THREE being available as a global?)
  2. replace npm run dev with a short Express script running express-middleware-rollup, so that each example script can be compiled on the fly when build/examples/js/foo.js is requested.
  3. on each release, run a script that compiles each example to ES5 with globals exports, and puts it into build/examples/*. On the docs page for each example file, we could add buttons to Download FooLoader.min.js for convenience. This way the threejs.org site should continue to work with older browsers.

Have to say I will be very happy when ES6 modules are widely supported and this is a non-issue. 😅

Contributor

Itee commented Oct 7, 2017

@looeee

Maybe you live in a country with good internet, but no, everybody in the world does not have a 5mb+ connection in 2017, and they still won't in 2018...

I'm totally agree with that fact, but if you're in this country you will got real pain to load any of the "exemple" display on the threejs.org wall too, isn't it ? Or load a simple model with a single texture under 3scd ?

@donmccurdy

I disagree on this. Three.js is used for many things, like generative art and data visualization, which do not need any models or textures. It is a general 3D library, not a game engine, and developers targeting mobile devices may reasonably want their page to be well under ~1MB in size.

I will be really surprise if you could make a full web site about generative art without any texture or image ( Simply a webgl viewport without any text, any register, any facebook/google/etc.. scripts, anythings in fact ? Maybe... but i think this is a really tiny amount of people on the world of the Art. ). Or it is a proof of concept ? In this case the fact that the app is not optimized is accepted...

A tool for data visualization... ok, and what about the data to load ? Are they inferior to 5mb ? Will you really make a complet webapp for simply 5Mb of data in your view ? I'm not sure.

The fact that Threejs is a general 3D library, and not a game engine, is already accepted by the community ! But about mobile device, first there browsers are up to date ( this is not IE7 ), second the mobile app developers will certainly optimize there code to get to best application footprint, and in this way will certainly use all the previous tools ( minification, tree shaking, ES6 module import etc... ) and not load three.min.js at all, but make their own compilation !

Finally, yes all the stuff in example folder is not to put in src OF COURSE !!! But, IMO, the major part should be !

The good question to ask could be: Which stuff need to be put in src right now, without breaking too many things ?

Contributor

looeee commented Oct 7, 2017

Yeah, that would be ideal. But I was unsure whether to have 2 files for each. People wouldn't know which one to contribute to.

@mrdoob I wasn't suggesting having two files - just the current files, make sure they can be converted with this script, and then people can run the script on any example they want to use as a module.

Contributor

Itee commented Oct 7, 2017

And what about the inverse ? Convert current example script to es6 modules and make a gulp/node/whatyouwant scripter to automate current example script creation? With a included warning for developers that would modify them instead of the es6 module ? This will keep backward compatibility, allow upgrading script to es6, and warn like deprecated stuff for developers... Furthermore this "scripter" could be remove later when it won't be required anymore.

Contributor

looeee commented Oct 7, 2017

@tristanvalcke as I understand it, the idea is to make things easier for beginners for whom having to understand both how a module bundling system and three.js work at the same time will be overwhelming.

Contributor

Itee commented Oct 7, 2017

Ok ! That more or less what i understood too.

And so... what is the best/easier for beginner ? @mrdoob

Have a single fully packed library with all could ever need the beginner to make is app ( which is currently not the case ) ?

Using a single line:

<script src="three.min.js"></script>

Or like actualy

Try to search everywhere in example where is located the piece of code that he want and will require time to find desired stuff and much more script src tags, like:

nota: this is like extending threejs by modules, isn't it ? But absolutely not like ES6 should be use, which will not facilitate his javascript learning in future, right ?

<script src="builds/three.min.js"></script>
<script src="example/js/controls/FirstPersonControls.js"></script> // <- not minimized !
<script src="example/js/effects/AnaglyphEffect.js"></script> // <- not minimized !
<script src="example/js/loaders/OBJLoader.js"></script> // <- not minimized !
<script src="example/js/GPUParticleSystem.js"></script> // <- not minimized  !
<script src="example/js/Detector.js"></script> // <- not minimized !
... and much more to search where looking for...

If i am a beginner, i think, i will prefer option 1 !

Much of the discussion would be simplified if we simply had the Three.js modules be exposed individually, using es6 modules. Not that I'm recommending that, it would fail whenever the code is reorganized. But a hybrid with three.module.js and all the examples as individual es6 modules would be OK.

Then we could have two branches of the examples tree, the current one, and one with them converted to es6 modules (which many of us have already done for those we use). I've certainly solved it for myself easily enough.

It might be possible to create a few bundles for the most common use, but that's a downward spiral and I definitely don't recommend it.

The good news is that the examples are so well formatted that the conversion would not be difficult, but the last time I counted, there were 177 *.js files under examples. I bet most of those could be converted with a bash script!

This is definitely a non-trivial problem. And there are higher priority issues such as webgl2.

Contributor

trusktr commented Oct 8, 2017 edited

I still think the easiest way to appease both groups is having two entry points.

Either

  1. Three.full.js (everything) and Three.js (same as current), or
  2. Three.js (the current one) and Three.extras.js (additional stuff that adds to THREE global, but built with rollup from modules just like Three.js).

All these other ideas of build steps duplicating files that we keep in the examples folders just seems like more burden and messy.

Contributor

trusktr commented Oct 8, 2017

Adding an extra entry point (and updating examples to use them) is the simplest: no new infrastructure to maintain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment