I’ve been thinking about how concept instances are configured and have written a draft note about this.
I’d love any help any of you can give me, from hard questions to new examples to better ideas…
Hi Daniel, this was a great read — I think you are onto something interesting.
Re; More extent examples; Here are some of my thoughts:
- In GitHub, the repository name is is associated with the organization or a user account. For example, kodless repo currently lives in an organization called
kodless-orgyet I can still create a completely new and different repository with the same name (i.e.,kodless) under my personal user account (i.e.,BarishNamazov/kodless). Contrast this with, for example, creating a bucket in AWS: the name of the bucket must be unique globally. Can you imagine how each of them did their concept design based on this fact? Perhaps AWS uses theCapabilityconcept? - In Google Drive, you can share the whole contents of a folder without sharing the folder. Then the receiving user still doesn’t have access to the folder (which makes sense from a security standpoint), or even a “virtual directory” where they can see these files together. Trying to conceptualize how Google permission system would be an interesting task. UNIX permissions would even be more interesting
Re; Lifting. This is interesting! I actually have done lifting before (without calling it “lifting”) and it has made my life much easier. I still need more thinking to decide when exactly lifting is good idea to do in practice (more about my hesitation in a bit), but let me explain my use case. I run a course management app for a few small education businesses (when I say course, it refers to the business). Now, in this app, there are bunch of simple functionality: creating students, classes, managing payments, notifications, etc, and each course should independently have these features (i.e., as if each course has their own separate app). One way to run this app for multiple courses would be running a separate server instance for each. That would require tons of maintenance and I don’t have time for that (e.g., I’d have to migrate N instances during updates). Another approach would be storing course in the state for pretty much every concept (Student, Class, Payment, CashRegister, Lesson, etc.), and that would be a lot of work to manage, not to mention an additional parameter passed to every single method. Instead, we just lift Course concept out of states of everything else, and to make things even simpler, I store the course id in the cookie, which means our friendly WebSession concept already tells me which concept instantiation to look at. Another nice benefit is, this allows me to store the data for each course in completely separate MongoDB collections (can also apply to tables if SQL is used), which resolves some security concerns as well (e.g., a bug where you return data from another course), and increases performance (e.g., just because one course might have huge number of students it won’t affect query cost a small course – it would otherwise even if you indexed based on course). The point is, this lifting makes managing multiple courses add almost zero overhead to my implementation, as if it was implemented for a single course (additionally, my concepts are now more generic and don’t depend on some Course concept as well). Here’s an example route:
@Router.delete("/class/:id/students/:studentId")
async removeStudentFromClass(session: WebSessionDoc, id: string, studentId: string) {
const { course, role } = WebSession.get(session);
if (RoleLevel[role] < PermissionLevel.Admin) {
throw new NotAllowedError();
}
const cId = new ObjectId(id);
const sId = new ObjectId(studentId);
const result = await Class[course].removeStudent(cId, sId);
await Tuition[course].removeTuition(cId, sId);
return result;
}
Notice how I just use Class[course] to refer to that instantiation of Class concept. Happy to answer more questions about this (DNJ, you should have access to this private repository – I gave you access in a different context).
Now, coming back to my hesitation about lifting, as great move it is — I think you’d agree with me that it’s not always intuitive to lift something up when we can. For example, would you lift the User concept from an Auth concept? Or, slightly more interestingly, would you lift the User concept from the Comment concept (to lose author from its state). Let’s go further (just playing the devil’s attorney here) and lift the generic Target concept from Comment concept as well – now it just contains a content: Comment -> one String which means we can get rid of this concept completely and just make them be [User] -> [Post] -> set String. We just deconceptualized our app After all, all data in the app is just relation of some objects, so in theory we can lift enough times to not have any concepts and only mappings (i.e., 0 modularity). Lifting, when done the right amount, seems to work wonders (and make app more modular) at the cost of “global functions” (e.g., postInChat in your example would previously be inside Chat concept), but doing it too much just loses the modularity.
All in all, very exciting stuff and we definitely should look more into this!
1 Like
Hi @barish: welcome to the forum, and thank you for this really helpful and insightful post!
Your example is a fantastic illustration of lifting, and your comments about separating the databases and the security and performance benefits of doing that are great. I wonder whether you could do the same thing with a SQL database.
You’re absolutely right that you could lift every type, but that would move all the functionality into the syncs, and as you put it would “deconceptualize” the app. The design principles that I have in mind are:
-
Simple OP. The operational principle (OP) of the concept should describe the essential functionality and no more and should not introduce context that is not necessary to understand the concept. So for example if you’re explaining the concept of an Order in an online store to a non-technical person you would talk about how when an order has been issued for an item it will later be delivered, etc, and you would have no need to say that the order is associated with the user, since that’s just implicit. But in contrast if you wanted to explain an Auction concept (eg, in eBay), the OP would have to mention the user, since what one user experiences depends on the behavior of other users (eg, if you get outbid).
-
Encapsulation of function. The code inside the concept should encapsulate the complexity of the functionality and especially anything likely to change. This is just a standard Parnas criterion. On the other side, the sync should have no concept-specific functionality in it, and should just do basic lookups to find instances (like your Class[course]). So taking your Auth example, if you made the (bad) argument that user authentication just required a lookup, and you had an indexed relation of the form
UserName -> Password -> Useryou would have to modify the lookup in the sync when you decided to hash your passwords; and then when you introduced salting, you’d have to modify it again. Even more basically you’d need to check that usernames are unique when created, and (I think) that even such basic uniqueness checks should be discouraged in syncs and are a sign that you shouldn’t have lifted.
So here’s my conclusion: lifting is a useful design move, but like all design moves needs to be applied with care. The nice thing about it is that it seems to be a move that can be evaluated entirely at the level of functionality without considering code (even though it has code implications). And as I mentioned in my piece, the lifting question can be a useful tool for discovering complications in the requirements.
@barish : replying to your opening examples.
Organizational context. The GitHub repo case suggests to me that some contexts might be artificial and purely for organizational purposes. Storing a repo in the context of a single user seems to be only because (a) it provides a shortcut for the user to find it (ie. naming); and (b) it lets the user prevent others from reading or writing it (ie, access control). Both of these reasons, naming and access control, are organizational in nature and are not relevant to the inherent functionality.
Functional context. In contrast, consider the Karma concept. Its very purpose is to determine whether a user should be allowed to do certain things depending on earlier behavior. So having a user in the context of a Karma instance is functional, not just organizational.
Symptom of organizational context. One symptom of a concept’s context being organizational is that the context isn’t always needed. I can put a (link to a) personal repo in a collection of repos somewhere, and it doesn’t matter when someone uses the repo that it originated in my personal repo. But it would make no sense to do that with a karma instance: any time you credit or spend karma points, you do it for a particular user.
So in both your cases, I think we have examples of organizational context. My gut feeling is that organizational context usually involves some concept functionality itself: a directory structure, a capability, etc.
In fact, I find the GitHub concepts for organizing repos super confusing: when I try and find a repo in an organization I belong to, it often doesn’t show in in my “top repo” list even if I access it often, and I haven’t been to figure out how to edit that list, which is incredibly annoying.
I wonder if the design considerations around context and concept extent are related to Andrew Hinton’s work on context.
See prior thread:
Thank you for the post Daniel. I would like to add my thoughts.
Concept Instances & Extents …
I understand your comments on concept instances to be all about distinguishing them from objects in OOP (or even abstract data types). This is worthwhile; in fact, I’ve been explicitly asked before, “are concepts not just abstract data types?” To that end, concepts are like a “service” (e.g., a label service responsible for mapping messages to labelObjects).
I think the class-like definitions of concepts might be what leads people to see them as just objects. Clearly they are not. You may implement a concept as an object or you may do it another way. We implement ours as serverless applications running in AWS. To that end, I think concept principles shine as guidance to tricky design problems, like those about service granularity.
Lifting: A Design Move…
I enjoyed your thoughts on lifting. Having application specific state feels natural and, as you point out, can simplify concept definitions.
It does make me think, though: if there is application state, then surely there are application actions to manipulate that state. The line between a concept and app blur.
In other conversations we’ve wrestled with whether concepts can be composed into other concepts. Perhaps the result of a lift operation leads to one of these composite concepts (like with the “group” concept).
Challenge to readers. Is there a neat way to factor this behavior out into a separate concept (whose purpose is to detect and prevent abuse), in the same way that the Capability concept was used to factor linking out of the Meeting concept?
A calendar or itinerary concept would do the trick. It would be a great place to implement constraints like those that prevent double booking. You might synchronize Calendar.createEvent with Reservation.make.
1 Like
Hi @nphair and welcome to the forum!
Your points about concepts and OOP are great: as you note, it’s possible to implement concepts in a non OOP setting and you’re doing that very effectively. I think describing concepts as being about service granularity is a very good way of putting it.
Regarding lifting and application state: I don’t think you do actually need application actions. If you did, then, as you note, it would introducing a kind of higher level concept. Perhaps that’s even a criterion for ensuring that the lifting is valid and isn’t taking significant behavior out of the concept that then needs to be embodied in an application action. Do you have an example of where you think an application action would be needed?
Interesting suggestion to use the Calendar concept to handle preventing people from booking two restaurants at once in an app like OpenTable or Resy. I’m not sure this is quite right, since presumably a calendar should let you have two events at the same time, and I wonder if the purpose and behavior of Calendar is in general well enough aligned with this rather specialized situation.
Regarding lifting and application state: …
Oh I think I misunderstood. The application state maintains relations between concept instances. Not data. Which is why you write “lift the meeting concept” and not “lift the [link state] from the meeting concept”.
The application relationships are then maintained by the synchronizations. No application actions needed.
Also, do you have more information on your formal concept notation? Your zoom example is definitely the most detailed version I’ve seen. It is very helpful.
On that note, in the startMeeting synch does m.ch suggest that the ch mapping is stored inside the concept (i.e., [Meeting -> Chat] and not [Meeting] -> [Chat])? Or am I reading too much into that?
Lastly, continuing with the “Challenge to readers” discussion, you’re right, Calendar is not quite it. I think a Schedule or Timetable gets you closer. The idea of a Schedule not supporting overlapping events would be familiar too. For instance, students building their semester schedule expect not to be able to enroll in 2 courses offered at the same time.
Yes, exactly: the application state maps concept instances. It might also be OK for the application state to relate concept instances to simple values (eg mapping a key to a concept instance). The essential idea is that these mappings just represent the configuration and not behavioral state.
Regarding your question about m.ch. We have this configuration decl in the state
let M = Meeting [U], C = Chat [U]
ch: M -> lone C
and in the sync
startMeeting (host: U, k: K, out m: M)
...
C.new (c)
m.ch := c // replaces on restart
This means that when the startMeeting sync happens, the configuration changes and the application state relation ch is updated so that m is mapped to c. That is, the chat associated with meeting m is now the new chat c. This reflects the fact that in Zoom whenever you restart a meeting you get a fresh chat (which is perhaps not a good design decision).
I haven’t written up this syntax properly but we’re hoping to add updates of this form to Alloy. The intuition I have in mind is that this is like a record update: the ch field of m is set to c. But the actual semantics is relational, so the formal meaning (in Alloy) is
ch' = ch ++ m -> c
(that is, the new value of the ch relation is the old value overwritten with the relation containing a single tuple that maps m to c).
Hi,
I wonder if we could keep the distinction between apps and concepts more clear, namely all mutable state is encapsulated in the concepts and apps are just static configurations of concepts. Otherwise I kind of agree with @nphair that the line between concept and app start to blur. In the example of the zoom app, if the Capability concept was generalised to have Key as a parameter maybe we could also use it also to maintain a relation between chats and meetings (or have a new concept for that purpose), and then the app actions could be pure synchronisation?
Best,
Alcino
Hi @alcino,
I don’t think that works because in some cases the configuration changes at runtime. And in many cases it’s set up at runtime. Even in the simplest version of the Zoom meeting case, the association between a Meeting and a Chat is dynamic because the Meeting is created on the fly. I prefer to think of configurations not as static but as changing with lower frequency.
Daniel
Couldn’t we lift the Reservation concept in two different ways, so that we have one instance per restaurant where users reserve tables and one instance per user where restaurants reserve user time slots?
1 Like
That’s a really clever idea! We should work out the details to see what that would mean in practice. One thing I like immediately about it is that it would handle the problem of one user making reservations for two different groups, by associating the reservation with the group and not the reserver. I also wonder how it might address some of the corner cases that annoy people about reservation systems (eg, not being able to book one restaurant for cocktails and then another for dinner).